nordic-dev.net/PAMGuard
nordic-dev.net/PAMGuard
2
0
Fork 0
mirror of https://github.com/PAMGuard/PAMGuard.git synced 2024-10-29 23:30:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'upstream/main'

Browse Source
This commit is contained in:
Jamie Mac 2024-04-29 09:26:40 +01:00
commit 17c428b870
129 changed files with 3211 additions and 747 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

183
README.html
View File

@ -387,8 +387,11 @@ PamguardBeta_ViewerMode.exe):</p>
<h1><a name="_LATEST_VERSION_2.02.03"></a><a name="_VERSION_2.02.07_January"></a><a
name="_Latest_Version_2.02.10"></a><em><span style='font-size:12.0pt;
font-family:"Cambria",serif;font-style:normal'><a
href="#_Version_2.02.10_January">Latest Version 2.02.10 January 2024</a></span></em></h1>
font-family:"Cambria",serif;font-style:normal'><a href="#_Version_2.02.11_April">Latest
Version 2.02.11 April 2024</a></span></em></h1>
<h1><em><span style='font-size:12.0pt;font-family:"Cambria",serif;font-style:
normal'><a href="#_Version_2.02.10_January">Version 2.02.10 January 2024</a></span></em></h1>
<h1><em><span style='font-size:12.0pt;font-family:"Cambria",serif;font-style:
normal'><a href="#_Version_2.02.09_June">Version 2.02.09 June 2023</a></span></em></h1>
@ -461,14 +464,47 @@ Version 2.00.10 June 2017</a></span></h1>
<h1><a name="_Latest_Version_2.02.03_1"></a><a name="_Latest_Version_2.02.05"></a><a
name="_Latest_Version_2.02.06"></a><a name="_Latest_Version_2.02.07"></a><a
name="_Latest_Version_2.02.08"></a><a name="_Version_2.02.09_June"></a><a
name="_Version_2.02.10_January"></a>Version 2.02.10 January 2024</h1>
name="_Version_2.02.10_January"></a><a name="_Version_2.02.11_April"></a>Version
2.02.11 April 2024</h1>
<p class=MsoNormal>&nbsp;</p>
<h2>New Features</h2>
<p class=MsoNormal>Click detector: Remembers locations of displays and doesn’t
continually reset them. </p>
<p class=MsoNormal>Help for Matched Click Classifier</p>
<h2>Bug Fixes</h2>
<p class=MsoNormal>Linking clicks to offline clicks table. We had a database
that had become corrupted so added code to relink offline clicks to their corresponding
clicks from binary data. </p>
<p class=MsoNormal>Drawing non-acoustic data: Data that were not associated
with any hydrophones, e.g. visual sightings in Logger forms were not drawing on
the map. This fixed and PAMGuard will use the vessels GPS position as
reference. </p>
<p class=MsoNormal>Lookup tables: fix feature which was causing table entries
to repeat. </p>
<p class=MsoNormal>Click Train Detector: Add exception handlers to avoid errors
as PAMGuard stops / restarts. </p>
<p class=MsoNormal>Group Detection starts and ends: Check data integrity
function fixed and now inserts correct times of start and ends of events into
database. </p>
<h1>Version 2.02.10 January 2024</h1>
<h2><span lang=EN-US>New Features</span></h2>
<p class=MsoNormal><b><span lang=EN-US>Importing modules</span></b><span
lang=EN-US> from other configurations: New options from file menu allowing
import of specific modules, or module settings from other configurations. E.g.
if you had three similar configurations and had set one of them up with a new
lang=EN-US> from other configurations: New options from file menu allowing import
of specific modules, or module settings from other configurations. E.g. if you
had three similar configurations and had set one of them up with a new
detector, or got the click classifier settings set up just right in one of
those configurations, you can import the additional modules or the click
detector settings easily into the other configurations. </span></p>
@ -496,9 +532,9 @@ correctly saving updated bearings to the database. Now fixed. </span></p>
<p class=MsoNormal><b><span lang=EN-US>ROCCA Classifier fixes</span></b></p>
<p class=MsoNormal><span lang=EN-US>Allow Rocca to run without classifiers:
Fixed bug that threw an error if no classifier files were &nbsp;specified in
Rocca Params dialog</span></p>
<p class=MsoNormal><span lang=EN-US>Allow Rocca to run without classifiers: Fixed
bug that threw an error if no classifier files were &nbsp;specified in Rocca
Params dialog</span></p>
<p class=MsoNormal><span lang=EN-US>Fix memory issue with
RoccaContourDataBlocks not being released for garbage collection</span></p>
@ -596,9 +632,9 @@ angle offsets applied to static hydrophones in viewer mode. This is now fixed. <
<p class=MsoNormal><span lang=EN-US>Click tool bar: Correctly shows event
selection options even if no species classification options are in place. </span></p>
<p class=MsoNormal><span lang=EN-US>Fixed Landmarks: Earlier versions were ‘losing’
these every time PAMGuard started or new data were loaded in viewer mode. Now
fixed. </span></p>
<p class=MsoNormal><span lang=EN-US>Fixed Landmarks: Earlier versions were
‘losing’ these every time PAMGuard started or new data were loaded in viewer
mode. Now fixed. </span></p>
<p class=MsoNormal><span lang=EN-US>ROCCA: Fixed (another) memory leak which
caused PAMGuard to crash when processing large data sets with the ROCCA
@ -767,8 +803,9 @@ the TF FX display to crash if no data were displayed.</span></p>
<p class=MsoNormal><span lang=EN-US>See major release notes for V 2.02.01
below. </span></p>
<p class=MsoNormal><span lang=EN-US>Bug 495: TD FX display throws NullPointerException
if user has removed all data units and then moves mouse over display area.</span></p>
<p class=MsoNormal><span lang=EN-US>Bug 495: TD FX display throws
NullPointerException if user has removed all data units and then moves mouse
over display area.</span></p>
<h1><a name="_Latest_Version_2.02.01"></a><span lang=EN-US>Version 2.02.01
October 2021</span></h1>
@ -791,11 +828,11 @@ font-family:"Times New Roman",serif'>&nbsp;</span></p>
<h2>File Format Change</h2>
<p class=MsoNormal>Changes have been made to the binary file format to support the
output of additional noise outputs for certain detectors (See below). Binary
files created with this version will not be compatible with earlier versions
2.01.### and below. This version will read and may convert earlier format
binary files.</p>
<p class=MsoNormal>Changes have been made to the binary file format to support
the output of additional noise outputs for certain detectors (See below).
Binary files created with this version will not be compatible with earlier
versions 2.01.### and below. This version will read and may convert earlier
format binary files.</p>
<p class=MsoNormal style='margin-bottom:0cm'><span style='font-size:12.0pt;
font-family:"Times New Roman",serif'>&nbsp;</span></p>
@ -1181,10 +1218,10 @@ lang=EN-US> </span>Add functionality for bluetooth headsets. </p>
<p class=MsoListParagraph style='text-indent:-18.0pt'><span lang=EN-US>2. </span><span
lang=EN-US style='font-size:7.0pt;font-family:"Times New Roman",serif'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span
lang=EN-US> </span>Add user-facing option to adjust the startup delay for the
time-correction (Global Time module). This provides a workaround to speed up
analysis of thousands of wav files (i.e. by setting startup delay to 0 instead
of default value of 2000 ms). </p>
lang=EN-US> </span>Add user-facing option to adjust the startup delay for the time-correction
(Global Time module). This provides a workaround to speed up analysis of
thousands of wav files (i.e. by setting startup delay to 0 instead of default
value of 2000 ms). </p>
<p class=MsoListParagraph style='text-indent:-18.0pt'><span lang=EN-US>3. </span><span
lang=EN-US style='font-size:7.0pt;font-family:"Times New Roman",serif'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span
@ -1415,9 +1452,9 @@ be compatible with this version, and vice-versa.</p>
<p class=MsoListParagraph style='text-indent:-18.0pt'><span lang=EN-US>2. </span><span
lang=EN-US style='font-size:7.0pt;font-family:"Times New Roman",serif'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span>
Java 12 is better at handling Windows scaling issues on high-DPI displays.
Beyond that, users should not notice much of a difference between this version
and previous beta releases.</p>
Java 12 is better at handling Windows scaling issues on high-DPI displays. Beyond
that, users should not notice much of a difference between this version and
previous beta releases.</p>
<!-- ************************************************************************************************************************** --><!-- ************************************************************************************************************************** -->
@ -1706,8 +1743,8 @@ with installation and use of this version.</span></p>
<p class=MsoListParagraph style='text-indent:-18.0pt'><span lang=EN-US>1. </span><span
lang=EN-US style='font-size:7.0pt;font-family:"Times New Roman",serif'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
</span>Bug 338. Problem displaying coastlines and bathymetric contours around the
dateline (+/- 180 longitude) in the map.</p>
</span>Bug 338. Problem displaying coastlines and bathymetric contours around
the dateline (+/- 180 longitude) in the map.</p>
<p class=MsoListParagraph style='text-indent:-18.0pt'><span lang=EN-US>2. </span><span
lang=EN-US style='font-size:7.0pt;font-family:"Times New Roman",serif'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
@ -2218,8 +2255,8 @@ UID
<p class=MsoNormal><span lang=EN-US>If old data are opened with the PAMGuard
viewer they will automatically be converted. For safety, the original binary
files will not be overwritten and the new data will be placed in a new folder
on your computer with the same path as the old data, but suffixed with ‘_WithUID’,
e.g. if your binary data were previously stored in the folder
on your computer with the same path as the old data, but suffixed with
‘_WithUID’, e.g. if your binary data were previously stored in the folder
C:\MySurvey\binarydata the new data will be written to
C:\MySurvey\binarydata_WithUID. </span></p>
@ -2295,8 +2332,8 @@ Hawaii/Temperate Pacific/North Atlantic datasets. This has been corrected.</p>
<p class=MsoListParagraph style='text-indent:-18.0pt'><span lang=EN-US>2.</span><span
lang=EN-US style='font-size:7.0pt;font-family:"Times New Roman",serif'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
</span>Bug 320. Pamguard stopped reading Click Detector Event data from database
when target motion analysis information was encountered. Corrected.</p>
</span>Bug 320. Pamguard stopped reading Click Detector Event data from
database when target motion analysis information was encountered. Corrected.</p>
<p class=MsoListParagraph style='text-indent:-18.0pt'><span lang=EN-US>3.</span><span
lang=EN-US style='font-size:7.0pt;font-family:"Times New Roman",serif'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
@ -2578,9 +2615,9 @@ main click detector display. </span></p>
<p class=MsoNormal><b><span lang=EN-US>Target Motion Analysis</span></b></p>
<p class=MsoNormal><span lang=EN-US>A major piece of work has been undertaken
to improve the Target Motion tracking with PAMGuard. Details are available in
the online help. Users of the Click Detector will notice the following changes:</span></p>
<p class=MsoNormal><span lang=EN-US>A major piece of work has been undertaken to
improve the Target Motion tracking with PAMGuard. Details are available in the
online help. Users of the Click Detector will notice the following changes:</span></p>
<p class=MsoListParagraph style='text-indent:-18.0pt'><span lang=EN-US>1.</span><span
lang=EN-US style='font-size:7.0pt;font-family:"Times New Roman",serif'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
@ -2752,8 +2789,8 @@ being imported into the new database. Problem 2 was that indexing of imported
click events in the new database was incorrect. Both these issues have now been
fixed. </span></p>
<h1><a name="_Latest_Version_1.15.02"></a><span lang=EN-US>Version 1.15.02 March
2016</span></h1>
<h1><a name="_Latest_Version_1.15.02"></a><span lang=EN-US>Version 1.15.02
March 2016</span></h1>
<p class=MsoNormal>A number of small bug fixes following release of 1.15.00.</p>
@ -2905,13 +2942,13 @@ for details. </span></p>
<p class=MsoListParagraph style='text-indent:-18.0pt'><span lang=EN-US>3.</span><span
lang=EN-US style='font-size:7.0pt;font-family:"Times New Roman",serif'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
</span><span lang=EN-US>GPS Loading into PAMGuard Viewer. This has been modified
so that the rules governing GPS data collection and storage also apply when
loading data from the database. For instance, if you've stored all GPS data,
you've probably got a record every second in the database which can create
memory overflows if you try to load a lot of data in the viewer. You can now
tell PAMGuard to only load a data point every n seconds which will reduce the
number of points loaded. Useful when making large scale overview maps of a
</span><span lang=EN-US>GPS Loading into PAMGuard Viewer. This has been
modified so that the rules governing GPS data collection and storage also apply
when loading data from the database. For instance, if you've stored all GPS
data, you've probably got a record every second in the database which can
create memory overflows if you try to load a lot of data in the viewer. You can
now tell PAMGuard to only load a data point every n seconds which will reduce
the number of points loaded. Useful when making large scale overview maps of a
survey. </span></p>
<p class=MsoListParagraph style='text-indent:-18.0pt'><span lang=EN-US>4.</span><span
@ -3018,8 +3055,8 @@ allocation to allow more memory for the database interface. Hopefully Fixed. </p
<p class=MsoListParagraph style='margin-left:38.25pt;text-indent:-20.25pt'><span
lang=EN-AU>9.</span><span lang=EN-AU style='font-size:7.0pt;font-family:"Times New Roman",serif'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
</span>Bug 239. <span lang=EN-AU>Fixed bug in the DIFAR module that was
incorrectly preventing cross-fixes for some calls.</span></p>
</span>Bug 239. <span lang=EN-AU>Fixed bug in the DIFAR module that was incorrectly
preventing cross-fixes for some calls.</span></p>
<p class=MsoNormal>Details of these bugs can be found at <a
href="https://sourceforge.net/p/pamguard/bugs">https://sourceforge.net/p/pamguard/bugs</a></p>
@ -3210,8 +3247,9 @@ to read on a time. Fixed</p>
<p class=MsoNormal><i>Menu Layout</i></p>
<p class=MsoNormal>The PAMGuard menus have been rearranged into a more intuitive
grouping which we believe will help users find functionality more easily. </p>
<p class=MsoNormal>The PAMGuard menus have been rearranged into a more
intuitive grouping which we believe will help users find functionality more
easily. </p>
<p class=MsoNormal>'Detection' menu has been renamed to 'Settings' since many
menu items within this menu were not directly to do with 'Detection'.</p>
@ -3302,9 +3340,9 @@ of third octave noise bands. See online help for details. </p>
<p class=MsoNormal><i>Filtered Noise Measurement</i> (Sound Processing Group)</p>
<p class=MsoNormal>This module, developed by Douglas Gillespie, measures noise levels
in a single frequency band using a variety of filter functions. See online help
for details. </p>
<p class=MsoNormal>This module, developed by Douglas Gillespie, measures noise
levels in a single frequency band using a variety of filter functions. See
online help for details. </p>
<p class=MsoNormal><i>Envelope Tracing</i> (Beta Only, Sound Processing Group)</p>
@ -3340,10 +3378,11 @@ different. Details are available in the online help. </p>
<p class=MsoNormal><i>FLAC File Support</i></p>
<p class=MsoNormal>Can now read raw audio data direct from FLAC files. <a
href="http://en.wikipedia.org/wiki/FLAC">FLAC</a> is a lossless compression algorithm
for audio data. Files, or folders of files are accessed in the same way as WAV
and AIFF files in the Sound Acquisition module. In a future release we also
hope to provide support for writing FLAC files from the sound recorder module. </p>
href="http://en.wikipedia.org/wiki/FLAC">FLAC</a> is a lossless compression
algorithm for audio data. Files, or folders of files are accessed in the same
way as WAV and AIFF files in the Sound Acquisition module. In a future release
we also hope to provide support for writing FLAC files from the sound recorder
module. </p>
<p class=MsoNormal><i>Sound Recorder Module</i></p>
@ -3476,9 +3515,9 @@ to these menus to provide additional information to users. &nbsp;</p>
<p class=MsoNormal style='margin-left:36.0pt'><i>Radar Display</i></p>
<p class=MsoNormal style='margin-left:36.0pt'>Functionality has been added to
the radar display so that bearings can be shown relative to either the vessel
or to true North. </p>
<p class=MsoNormal style='margin-left:36.0pt'>Functionality has been added to the
radar display so that bearings can be shown relative to either the vessel or to
true North. </p>
<p class=MsoNormal style='margin-left:36.0pt'>Better control of data in viewer
mode, making is easy to scroll through and view data for short time periods. </p>
@ -4116,8 +4155,8 @@ Symbol'>'</span><span style='font-size:7.0pt;font-family:"Times New Roman",serif
<p class=MsoListParagraph style='text-indent:-18.0pt'><span style='font-family:
Symbol'>'</span><span style='font-size:7.0pt;font-family:"Times New Roman",serif'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
</span>This results in a major speed up of data exchange between modules and can
lead to a x4 improvement in overall performance. </p>
</span>This results in a major speed up of data exchange between modules and
can lead to a x4 improvement in overall performance. </p>
<p class=MsoListParagraph style='text-indent:-18.0pt'><span style='font-family:
Symbol'>'</span><span style='font-size:7.0pt;font-family:"Times New Roman",serif'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
@ -4230,9 +4269,9 @@ Symbol'>'</span><span style='font-size:7.0pt;font-family:"Times New Roman",serif
<p class=MsoListParagraph style='text-indent:-18.0pt'><span style='font-family:
Symbol'>'</span><span style='font-size:7.0pt;font-family:"Times New Roman",serif'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
</span>Channel lists in output data streams of Decimator and other modules fixed,
so that when channel numbers change, downstream modules configurations get the
correct list of available channels. </p>
</span>Channel lists in output data streams of Decimator and other modules
fixed, so that when channel numbers change, downstream modules configurations
get the correct list of available channels. </p>
<p class=MsoListParagraph style='text-indent:-18.0pt'><span style='font-family:
Symbol'>'</span><span style='font-size:7.0pt;font-family:"Times New Roman",serif'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
@ -4263,11 +4302,11 @@ Symbol'>'</span><span style='font-size:7.0pt;font-family:"Times New Roman",serif
<p class=MsoListParagraph style='text-indent:-18.0pt'><span style='font-family:
Symbol'>'</span><span style='font-size:7.0pt;font-family:"Times New Roman",serif'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
</span>New menu functionality by right clicking on any of the tabs of the main
tab control will allow the user to copy the tab contents to the system
clipboard from where it can be copied into other programs (e.g. Word,
Powerpoint, etc.).Some modules, such as the map, have this implemented in other
menus (right click) and also allow printing.&nbsp; </p>
</span>New menu functionality by right clicking on any of the tabs of the main tab
control will allow the user to copy the tab contents to the system clipboard
from where it can be copied into other programs (e.g. Word, Powerpoint,
etc.).Some modules, such as the map, have this implemented in other menus
(right click) and also allow printing.&nbsp; </p>
<p class=MsoListParagraph style='text-indent:-18.0pt'><span style='font-family:
Symbol'>'</span><span style='font-size:7.0pt;font-family:"Times New Roman",serif'>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
@ -4335,8 +4374,8 @@ online help. </p>
<p class=MsoNormal><b>PAMGUARD Mixed Mode operation</b></p>
<p class=MsoNormal>Analyses data from wav or AIF file and synchronises it with
GPS data reloaded from a database so that detected sounds may be correctly
<p class=MsoNormal>Analyses data from wav or AIF file and synchronises it with GPS
data reloaded from a database so that detected sounds may be correctly
localised. Multiple display frames - enables PAMGUARD GUI to be split into
multiple display windows, displayed on multiple monitors if desired. Enables
the operator to simultaneously view the map and the click detector for example,
@ -4855,8 +4894,8 @@ Symbol'>'</span><span style='font-size:7.0pt;font-family:"Times New Roman",serif
<p class=MsoNormal>&nbsp;</p>
<p class=MsoNormal><a name="_Toc312065304"></a><a name="_Toc312063949"></a><span
class=Heading2Char><span style='font-size:13.0pt'>1.0Beta 22 Jan 2008 - Pamguard
starts two releases, core and beta release</span></span>, </p>
class=Heading2Char><span style='font-size:13.0pt'>1.0Beta 22 Jan 2008 -
Pamguard starts two releases, core and beta release</span></span>, </p>
<p class=MsoNormal>this is the beta release</p>

4
repo/pamguard/org/x3/2.2.6/_remote.repositories Normal file
View File

@ -0,0 +1,4 @@
#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
#Wed Nov 15 12:43:42 GMT 2023
x3-2.2.6.jar>=
x3-2.2.6.pom>=

BIN
repo/pamguard/org/x3/2.2.6/x3-2.2.6.jar Normal file
View File

Binary file not shown.

15
repo/pamguard/org/x3/2.2.6/x3-2.2.6.pom Normal file
View File

@ -0,0 +1,15 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<<<<<<<< HEAD:repo/tethys/org/javaclient/3.0/javaclient-3.0.pom
<groupId>tethys.org</groupId>
<artifactId>javaclient</artifactId>
<version>3.0</version>
========
<groupId>pamguard.org</groupId>
<artifactId>x3</artifactId>
<version>2.2.6</version>
>>>>>>>> upstream/main:repo/pamguard/org/x3/2.2.6/x3-2.2.6.pom
<description>POM was created from install:install-file</description>
</project>

4
repo/pamguard/org/x3/2.2.7/_remote.repositories Normal file
View File

@ -0,0 +1,4 @@
#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
#Fri Jan 12 10:06:13 GMT 2024
x3-2.2.7.jar>=
x3-2.2.7.pom>=

11
repo/pamguard/org/x3/2.2.7/m2e-lastUpdated.properties Normal file
View File

@ -0,0 +1,11 @@
#Fri Jan 12 10:09:16 GMT 2024
bedatadriven|https\://nexus.bedatadriven.com/content/groups/public/|javadoc=1705054156349
repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo|sources=1705054009341
repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo|javadoc=1705054156349
talan|https\://nexus.talanlabs.com/content/repositories/releases/|javadoc=1705054156349
central|https\://repo1.maven.org/maven2|sources=1705054009341
unidata-all|https\://artifacts.unidata.ucar.edu/repository/unidata-all/|javadoc=1705054156349
talan|https\://nexus.talanlabs.com/content/repositories/releases/|sources=1705054009341
bedatadriven|https\://nexus.bedatadriven.com/content/groups/public/|sources=1705054009341
unidata-all|https\://artifacts.unidata.ucar.edu/repository/unidata-all/|sources=1705054009341
central|https\://repo1.maven.org/maven2|javadoc=1705054156349

12
repo/pamguard/org/x3/2.2.7/x3-2.2.7-javadoc.jar.lastUpdated Normal file
View File

@ -0,0 +1,12 @@
#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
#Fri Jan 12 10:09:16 GMT 2024
@default-talan-https\://nexus.talanlabs.com/content/repositories/releases/.lastUpdated=1705054156080
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo/.lastUpdated=1705054154613
https\://repo1.maven.org/maven2/.error=
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo/.error=
https\://artifacts.unidata.ucar.edu/repository/unidata-all/.error=
https\://nexus.bedatadriven.com/content/groups/public/.error=
https\://nexus.bedatadriven.com/content/groups/public/.lastUpdated=1705054156060
https\://artifacts.unidata.ucar.edu/repository/unidata-all/.lastUpdated=1705054155868
https\://repo1.maven.org/maven2/.lastUpdated=1705054156344
https\://nexus.talanlabs.com/content/repositories/releases/.error=Could not transfer artifact pamguard.org\:x3\:jar\:javadoc\:2.2.7 from/to talan (https\://nexus.talanlabs.com/content/repositories/releases/)\: nexus.talanlabs.com

12
repo/pamguard/org/x3/2.2.7/x3-2.2.7-sources.jar.lastUpdated Normal file
View File

@ -0,0 +1,12 @@
#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
#Fri Jan 12 10:06:49 GMT 2024
@default-talan-https\://nexus.talanlabs.com/content/repositories/releases/.lastUpdated=1705054009062
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo/.lastUpdated=1705054007956
https\://repo1.maven.org/maven2/.error=
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo/.error=
https\://artifacts.unidata.ucar.edu/repository/unidata-all/.error=
https\://nexus.bedatadriven.com/content/groups/public/.error=
https\://nexus.bedatadriven.com/content/groups/public/.lastUpdated=1705054008945
https\://artifacts.unidata.ucar.edu/repository/unidata-all/.lastUpdated=1705054008751
https\://repo1.maven.org/maven2/.lastUpdated=1705054009308
https\://nexus.talanlabs.com/content/repositories/releases/.error=Could not transfer artifact pamguard.org\:x3\:jar\:sources\:2.2.7 from/to talan (https\://nexus.talanlabs.com/content/repositories/releases/)\: nexus.talanlabs.com

BIN
repo/pamguard/org/x3/2.2.7/x3-2.2.7.jar Normal file
View File

Binary file not shown.

15
repo/pamguard/org/x3/2.2.7/x3-2.2.7.pom Normal file
View File

@ -0,0 +1,15 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<<<<<<<< HEAD:repo/tethys/org/nilus/3.0/nilus-3.0.pom
<groupId>tethys.org</groupId>
<artifactId>nilus</artifactId>
<version>3.0</version>
========
<groupId>pamguard.org</groupId>
<artifactId>x3</artifactId>
<version>2.2.7</version>
>>>>>>>> upstream/main:repo/pamguard/org/x3/2.2.7/x3-2.2.7.pom
<description>POM was created from install:install-file</description>
</project>

16
repo/tethys/org/javaclient/3.0/javaclient-3.0-javadoc.jar.lastUpdated Normal file
View File

@ -0,0 +1,16 @@
#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
#Thu Mar 07 12:09:00 GMT 2024
@default-talan-https\://nexus.talanlabs.com/content/repositories/releases/.lastUpdated=1705320262317
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo/.lastUpdated=1705320259110
https\://repo1.maven.org/maven2/.error=
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo/.error=
https\://artifacts.unidata.ucar.edu/repository/unidata-all/.error=
https\://nexus.bedatadriven.com/content/groups/public/.error=
https\://artifacts.unidata.ucar.edu/repository/unidata-all/.lastUpdated=1705320260572
https\://repo1.maven.org/maven2/.lastUpdated=1705320262598
https\://nexus.talanlabs.com/content/repositories/releases/.error=Could not transfer artifact tethys.org\:javaclient\:jar\:javadoc\:3.0 from/to talan (https\://nexus.talanlabs.com/content/repositories/releases/)\: nexus.talanlabs.com
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo/.lastUpdated=1705569305037
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG_2/repo/.error=
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo/.error=
https\://nexus.bedatadriven.com/content/groups/public/.lastUpdated=1705320262257
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG_2/repo/.lastUpdated=1709813340827

16
repo/tethys/org/javaclient/3.0/javaclient-3.0-sources.jar.lastUpdated Normal file
View File

@ -0,0 +1,16 @@
#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
#Thu Mar 07 12:08:16 GMT 2024
@default-talan-https\://nexus.talanlabs.com/content/repositories/releases/.lastUpdated=1703957320119
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo/.error=
https\://repo1.maven.org/maven2/.error=
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo/.lastUpdated=1705074190844
https\://artifacts.unidata.ucar.edu/repository/unidata-all/.error=
https\://nexus.bedatadriven.com/content/groups/public/.error=
https\://artifacts.unidata.ucar.edu/repository/unidata-all/.lastUpdated=1703957319870
https\://repo1.maven.org/maven2/.lastUpdated=1703957320395
https\://nexus.talanlabs.com/content/repositories/releases/.error=Could not transfer artifact tethys.org\:javaclient\:jar\:sources\:3.0 from/to talan (https\://nexus.talanlabs.com/content/repositories/releases/)\: nexus.talanlabs.com
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo/.lastUpdated=1703957318729
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG_2/repo/.error=
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo/.error=
https\://nexus.bedatadriven.com/content/groups/public/.lastUpdated=1703957320102
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG_2/repo/.lastUpdated=1709813296623

6
repo/tethys/org/javaclient/3.0/javaclient-3.0.pom
View File

@ -2,8 +2,14 @@
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<<<<<<<< HEAD:repo/tethys/org/javaclient/3.0/javaclient-3.0.pom
<groupId>tethys.org</groupId>
<artifactId>javaclient</artifactId>
<version>3.0</version>
========
<groupId>pamguard.org</groupId>
<artifactId>x3</artifactId>
<version>2.2.6</version>
>>>>>>>> upstream/main:repo/pamguard/org/x3/2.2.6/x3-2.2.6.pom
<description>POM was created from install:install-file</description>
</project>

15
repo/tethys/org/javaclient/3.0/m2e-lastUpdated.properties Normal file
View File

@ -0,0 +1,15 @@
#Thu Mar 07 12:09:00 GMT 2024
bedatadriven|https\://nexus.bedatadriven.com/content/groups/public/|javadoc=1709813340828
repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo|javadoc=1705320262599
repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG_2/repo|javadoc=1709813340828
repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo|javadoc=1705569305040
talan|https\://nexus.talanlabs.com/content/repositories/releases/|sources=1709813296626
repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG_2/repo|sources=1709813296626
bedatadriven|https\://nexus.bedatadriven.com/content/groups/public/|sources=1709813296626
unidata-all|https\://artifacts.unidata.ucar.edu/repository/unidata-all/|sources=1709813296626
repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo|sources=1703957320396
repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo|sources=1705074190848
talan|https\://nexus.talanlabs.com/content/repositories/releases/|javadoc=1709813340828
central|https\://repo1.maven.org/maven2|sources=1709813296626
unidata-all|https\://artifacts.unidata.ucar.edu/repository/unidata-all/|javadoc=1709813340828
central|https\://repo1.maven.org/maven2|javadoc=1709813340828

10
repo/tethys/org/nilus/3.0/_remote.repositories
View File

@ -1,12 +1,4 @@
#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
<<<<<<<< HEAD:repo/org/pamguard/x3/2.2.5/_remote.repositories
#Wed Aug 02 09:25:44 BST 2023
X3-2.2.5-sources.jar>=
X3-2.2.5-javadoc.jar>=
X3-2.2.5.jar>=
X3-2.2.5.pom>=
========
#Thu Dec 21 11:14:13 GMT 2023
#Thu Mar 07 11:55:57 GMT 2024
nilus-3.0.pom>=
nilus-3.0.jar>=
>>>>>>>> upstream/main:repo/tethys/org/nilus/3.0/_remote.repositories

22
repo/tethys/org/nilus/3.0/m2e-lastUpdated.properties
View File

@ -1,11 +1,15 @@
#Thu Dec 21 16:45:12 GMT 2023
bedatadriven|https\://nexus.bedatadriven.com/content/groups/public/|javadoc=1703177112968
talan|https\://nexus.talanlabs.com/content/repositories/releases/|javadoc=1703177112968
#Thu Jan 18 09:15:05 GMT 2024
bedatadriven|https\://nexus.bedatadriven.com/content/groups/public/|javadoc=1705569305022
repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo|javadoc=1705320259098
repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG_2/repo|javadoc=1703177112968
central|https\://repo1.maven.org/maven2|sources=1703157324238
unidata-all|https\://artifacts.unidata.ucar.edu/repository/unidata-all/|javadoc=1703177112968
repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo|javadoc=1705569305022
repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG_2/repo|sources=1703157324238
talan|https\://nexus.talanlabs.com/content/repositories/releases/|sources=1703157324238
bedatadriven|https\://nexus.bedatadriven.com/content/groups/public/|sources=1703157324238
unidata-all|https\://artifacts.unidata.ucar.edu/repository/unidata-all/|sources=1703157324238
central|https\://repo1.maven.org/maven2|javadoc=1703177112968
talan|https\://nexus.talanlabs.com/content/repositories/releases/|sources=1705074190830
bedatadriven|https\://nexus.bedatadriven.com/content/groups/public/|sources=1705074190830
unidata-all|https\://artifacts.unidata.ucar.edu/repository/unidata-all/|sources=1705074190830
repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo|sources=1703957318711
repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo|sources=1705074190830
talan|https\://nexus.talanlabs.com/content/repositories/releases/|javadoc=1705569305022
central|https\://repo1.maven.org/maven2|sources=1705074190830
unidata-all|https\://artifacts.unidata.ucar.edu/repository/unidata-all/|javadoc=1705569305022
central|https\://repo1.maven.org/maven2|javadoc=1705569305022

12
repo/tethys/org/nilus/3.0/nilus-3.0-javadoc.jar.lastUpdated
View File

@ -1,12 +1,16 @@
#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
#Thu Dec 21 16:45:12 GMT 2023
#Thu Jan 18 09:15:05 GMT 2024
@default-talan-https\://nexus.talanlabs.com/content/repositories/releases/.lastUpdated=1703177112601
https\://repo1.maven.org/maven2/.error=
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG_2/repo/.error=
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG_2/repo/.lastUpdated=1703177110940
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo/.lastUpdated=1705320259096
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo/.error=
https\://artifacts.unidata.ucar.edu/repository/unidata-all/.error=
https\://nexus.bedatadriven.com/content/groups/public/.error=
https\://nexus.bedatadriven.com/content/groups/public/.lastUpdated=1703177112519
https\://artifacts.unidata.ucar.edu/repository/unidata-all/.lastUpdated=1703177112101
https\://repo1.maven.org/maven2/.lastUpdated=1703177112965
https\://nexus.talanlabs.com/content/repositories/releases/.error=Could not transfer artifact tethys.org\:nilus\:jar\:javadoc\:3.0 from/to talan (https\://nexus.talanlabs.com/content/repositories/releases/)\: nexus.talanlabs.com
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG_2/repo/.error=
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo/.lastUpdated=1705569305018
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo/.error=
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG_2/repo/.lastUpdated=1703177110940
https\://nexus.bedatadriven.com/content/groups/public/.lastUpdated=1703177112519

12
repo/tethys/org/nilus/3.0/nilus-3.0-sources.jar.lastUpdated
View File

@ -1,12 +1,16 @@
#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
#Thu Dec 21 11:15:24 GMT 2023
#Fri Jan 12 15:43:10 GMT 2024
@default-talan-https\://nexus.talanlabs.com/content/repositories/releases/.lastUpdated=1703157323819
https\://repo1.maven.org/maven2/.error=
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG_2/repo/.error=
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG_2/repo/.lastUpdated=1703157322932
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo/.error=
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo/.lastUpdated=1705074190824
https\://artifacts.unidata.ucar.edu/repository/unidata-all/.error=
https\://nexus.bedatadriven.com/content/groups/public/.error=
https\://nexus.bedatadriven.com/content/groups/public/.lastUpdated=1703157323770
https\://artifacts.unidata.ucar.edu/repository/unidata-all/.lastUpdated=1703157323508
https\://repo1.maven.org/maven2/.lastUpdated=1703157324237
https\://nexus.talanlabs.com/content/repositories/releases/.error=Could not transfer artifact tethys.org\:nilus\:jar\:sources\:3.0 from/to talan (https\://nexus.talanlabs.com/content/repositories/releases/)\: nexus.talanlabs.com
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG_2/repo/.error=
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo/.lastUpdated=1703957318705
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo/.error=
file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG_2/repo/.lastUpdated=1703157322932
https\://nexus.bedatadriven.com/content/groups/public/.lastUpdated=1703157323770

BIN
repo/tethys/org/nilus/3.0/nilus-3.0.jar
View File

Binary file not shown.

6
repo/tethys/org/nilus/3.0/nilus-3.0.pom
View File

@ -2,8 +2,14 @@
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<<<<<<<< HEAD:repo/tethys/org/nilus/3.0/nilus-3.0.pom
<groupId>tethys.org</groupId>
<artifactId>nilus</artifactId>
<version>3.0</version>
========
<groupId>pamguard.org</groupId>
<artifactId>x3</artifactId>
<version>2.2.7</version>
>>>>>>>> upstream/main:repo/pamguard/org/x3/2.2.7/x3-2.2.7.pom
<description>POM was created from install:install-file</description>
</project>

2
repo/tethys/org/nilus/maven-metadata-local.xml
View File

@ -7,6 +7,6 @@
<versions>
<version>3.0</version>
</versions>
<lastUpdated>20231221111413</lastUpdated>
<lastUpdated>20240307115557</lastUpdated>
</versioning>
</metadata>

1
src/Acquisition/AcquisitionControl.java
View File

@ -878,6 +878,7 @@ public class AcquisitionControl extends RawInputControlledUnit implements PamSet
public InputStoreInfo getStoreInfo(boolean detail) {
return getDaqProcess().getStoreInfo(detail);
}
@Override
public boolean setAnalysisStartTime(long startTime) {
return getDaqProcess().setAnalysisStartTime(startTime);

4
src/Acquisition/AcquisitionProcess.java
View File

@ -65,7 +65,7 @@ import pamScrollSystem.ViewLoadObserver;
* @see PamguardMVC.PamDataUnit
*
*/
public class AcquisitionProcess extends PamProcess implements DataInputStore {
public class AcquisitionProcess extends PamProcess {
public static final int LASTDATA = 2; // don't use zero since need to see if no notification has been received.
@ -1240,7 +1240,6 @@ public class AcquisitionProcess extends PamProcess implements DataInputStore {
return daqStatusDataBlock;
}
@Override
public InputStoreInfo getStoreInfo(boolean detail) {
if (runningSystem instanceof DataInputStore) {
return ((DataInputStore) runningSystem).getStoreInfo(detail);
@ -1250,7 +1249,6 @@ public class AcquisitionProcess extends PamProcess implements DataInputStore {
}
}
@Override
public boolean setAnalysisStartTime(long startTime) {
if (runningSystem instanceof DataInputStore) {
return ((DataInputStore) runningSystem).setAnalysisStartTime(startTime);

4
src/Acquisition/FileInputSystem.java
View File

@ -814,6 +814,10 @@ public class FileInputSystem extends DaqSystem implements ActionListener, PamSe
protected void collectFlacData() {
FileInputStream fileStream;
try {
File currFile = getCurrentFile();
if (currFile == null) {
return;
}
fileStream = new FileInputStream(getCurrentFile());
} catch (FileNotFoundException e) {
e.printStackTrace();

3
src/Acquisition/FolderInputSystem.java
View File

@ -602,6 +602,7 @@ public class FolderInputSystem extends FileInputSystem implements PamSettings, D
@Override
public File getCurrentFile() {
//System.out.println("All files: " + allFiles);
// System.out.printf("Folder: getCurrentfile. on %d of %d\n", currentFile, allFiles.size());
if (allFiles != null && allFiles.size() > currentFile) {
return allFiles.get(currentFile);
}
@ -689,7 +690,7 @@ public class FolderInputSystem extends FileInputSystem implements PamSettings, D
}
if (currentFile < allFiles.size()) {
// only restart if the file ended - not if it stopped
if (getStreamStatus() == STREAM_ENDED) {
if (getStreamStatus() == STREAM_ENDED && PamController.getInstance().isManualStop() == false) {
// System.out.println(String.format("Start new file timer (file %d/%d)",currentFile+1,allFiles.size()));
newFileTimer.start();
}

44
src/Array/ArrayManager.java
View File

@ -21,6 +21,7 @@ import Array.sensors.swing.ArraySensorPanelProvider;
import GPS.GPSControl;
import GPS.GPSDataBlock;
import GPS.GpsData;
import GPS.GpsDataUnit;
import PamController.PamControlledUnit;
import PamController.PamControlledUnitGUI;
import PamController.PamControlledUnitSettings;
@ -28,6 +29,7 @@ import PamController.PamController;
import PamController.PamGUIManager;
import PamController.PamSettingManager;
import PamController.PamSettings;
import PamController.masterReference.MasterReferencePoint;
import PamController.positionreference.PositionReference;
import PamModel.PamModuleInfo;
import PamUtils.PamUtils;
@ -989,6 +991,7 @@ public class ArrayManager extends PamControlledUnit implements PamSettings, PamO
* @return geometry data.
*/
public SnapshotGeometry getSnapshotGeometry(int hydrophoneMap, long timeMillis) {
PamArray currentArray = getCurrentArray();
if (currentArray == null) {
return null;
@ -1081,14 +1084,45 @@ public class ArrayManager extends PamControlledUnit implements PamSettings, PamO
}
}
if (nGood > 0) for (int p = 0; p < 3; p++) {
centre[p] /= nGood;
if (nGood > 0) {
for (int p = 0; p < 3; p++) {
centre[p] /= nGood;
}
return new SnapshotGeometry(currentArray, timeMillis, streamerList, hydrophoneList, firstStreamerPos,
new PamVector(centre), geometry, streamerError, hydrophoneError);
}
return new SnapshotGeometry(currentArray, timeMillis, streamerList, hydrophoneList, firstStreamerPos,
new PamVector(centre), geometry, streamerError, hydrophoneError);
else {
return getMasterReferenceGeometry(timeMillis);
}
}
/**
* Create a snapshot geometry from the master reference position, which will either be the GPS data, or
* the centre point of the array. Worst case is that it ends up as 0,0,0
* @param timeMillis
* @return
*/
private SnapshotGeometry getMasterReferenceGeometry(long timeMillis) {
GPSControl gpsControl = GPSControl.getGpsControl();
GpsData referencePoint = null;
if (gpsControl != null) {
GpsDataUnit shipPos = gpsControl.getShipPosition(timeMillis, true);
if (shipPos != null) {
referencePoint = shipPos.getGpsData();
}
}
if (referencePoint == null && MasterReferencePoint.getFixTime() != null && MasterReferencePoint.getLatLong() != null) {
// running out of options, so fall back to the master reference point, interpolated (probably has zero speeed)
referencePoint = new GpsData(MasterReferencePoint.getFixTime(), MasterReferencePoint.getLatLong()).getPredictedGPSData(timeMillis);
}
if (referencePoint == null) {
return null;
}
SnapshotGeometry snapgeom = new SnapshotGeometry(getCurrentArray(), timeMillis, null, null, referencePoint, new PamVector(0,0,0), null, null, null);
return snapgeom;
}
//
// public SnapshotGeometry getSubDetectionGeometry(PamDataUnit superDataUnit) {
// /**

65
src/GPS/GPSControl.java
View File

@ -4,6 +4,7 @@ import java.awt.Frame;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.io.Serializable;
import java.util.ListIterator;
import javax.swing.JMenu;
import javax.swing.JMenuItem;
@ -22,6 +23,7 @@ import PamController.PamSettings;
import PamController.positionreference.PositionReference;
import PamUtils.PamCalendar;
import PamView.dialog.warn.WarnOnce;
import PamguardMVC.PamDataBlock;
public class GPSControl extends PamControlledUnit implements PamSettings, PositionReference {
@ -242,9 +244,71 @@ public class GPSControl extends PamControlledUnit implements PamSettings, Positi
return super.removeUnit();
}
/**
* Gets the closest position based on time. No interpolation
* @param timeMilliseconds
* @return closest ship position based on time.
*/
public GpsDataUnit getShipPosition(long timeMilliseconds) {
return getGpsDataBlock().getClosestUnitMillis(timeMilliseconds);
}
/**
* Gets the closest position based on time.
* @param timeMilliseconds time in milliseconds
* @param interpolate interpolate between the point before and the one after.
* @return interpolated gps position.
*/
public GpsDataUnit getShipPosition(long timeMilliseconds, boolean interpolate) {
if (interpolate == false) {
return getGpsDataBlock().getClosestUnitMillis(timeMilliseconds);
}
// otherwise try to fine a point either side and weighted mean them or extrapolate.
GpsDataUnit pointBefore = null, pointAfter = null;
synchronized (getGpsDataBlock().getSynchLock()) {
ListIterator<GpsDataUnit> iter = getGpsDataBlock().getListIterator(timeMilliseconds, 0, PamDataBlock.MATCH_BEFORE, PamDataBlock.POSITION_BEFORE);
if (iter == null) {
return null;
}
if (iter.hasNext()) {
pointBefore = iter.next();
}
if (iter.hasNext()) {
pointAfter = iter.next();
}
}
if (pointBefore == null && pointAfter == null) {
return null;
}
if (pointBefore != null & pointAfter != null) {
// interpolate since we have them both.
double tB = timeMilliseconds-pointBefore.getTimeMilliseconds();
double tA = pointAfter.getTimeMilliseconds()-timeMilliseconds;
double wB = tA/(tB+tA); // weight for data before
double wA = tB/(tB+tA); // wedight for data after
double newLat = pointBefore.getGpsData().getLatitude()*wB + pointAfter.getGpsData().getLatitude()*wA;
double newLon = pointBefore.getGpsData().getLongitude()*wB + pointAfter.getGpsData().getLongitude()*wA;
// interpolating heading is more faff:
double hB = pointBefore.getGpsData().getHeading();
double hA = pointAfter.getGpsData().getHeading();
double hX = Math.sin(Math.toRadians(hB))*wB + Math.sin(Math.toRadians(hB))*wB;
double hY = Math.cos(Math.toRadians(hB))*wB + Math.cos(Math.toRadians(hB))*wB;
double newHead = Math.toDegrees(Math.atan2(hX, hY));
GpsData newData = new GpsData(newLat, newLon, 0, timeMilliseconds);
newData.setTrueHeading(newHead);
return new GpsDataUnit(timeMilliseconds, newData);
}
else if (pointBefore != null) {
// relatively common if we enter data just after a position was received.
GpsData newPos = pointBefore.getGpsData().getPredictedGPSData(timeMilliseconds);
return new GpsDataUnit(timeMilliseconds, newPos);
}
else if (pointAfter != null) {
GpsData newPos = pointAfter.getGpsData().getPredictedGPSData(timeMilliseconds);
return new GpsDataUnit(timeMilliseconds, newPos);
}
return null; // should never happen since one of the above 4 clauses must be met.
}
/**
* Do we want this string ? It will be either RMC or GGA and may want wildcarding
* @param stringId
@ -322,6 +386,7 @@ public class GPSControl extends PamControlledUnit implements PamSettings, Positi
public GPSDataMatcher getGpsDataMatcher() {
return gpsDataMatcher;
}
@Override
public GpsData getReferencePosition(long timeMillis) {
GpsDataUnit gpDU = getShipPosition(timeMillis);

7
src/PamController/DataInputStore.java
View File

@ -28,4 +28,11 @@ public interface DataInputStore {
*/
public boolean setAnalysisStartTime(long startTime);
/**
* Very specific command handler for batch status which will only work
* with the acquisition folderinputSystem or the tritech file processing.
* @return
*/
public String getBatchStatus();
}

12
src/PamController/DataIntegrityChecker.java Normal file
View File

@ -0,0 +1,12 @@
package PamController;
/**
* Provides a set of functions that can check and repair data.
* @author dg50
*
*/
public interface DataIntegrityChecker {
public boolean checkDataStore();
}

7
src/PamController/DataOutputStore.java
View File

@ -29,4 +29,11 @@ public interface DataOutputStore extends OfflineDataStore {
*/
public boolean deleteDataFrom(long timeMillis);
/**
* Get a data integrity checker. This can be called at startup to see if there is a problem.
* @return
*/
public DataIntegrityChecker getInegrityChecker();
}

15
src/PamController/PamConfiguration.java
View File

@ -170,12 +170,7 @@ public class PamConfiguration {
public PamDataBlock getDataBlock(Class blockType, String name) {
if (name == null) return null;
ArrayList<PamDataBlock> blocks = getDataBlocks(blockType, true);
System.out.println(name);
for (PamDataBlock dataBlock:blocks) {
// System.out.println(dataBlock.getLongDataName() + " ||| " + dataBlock.toString());
if (name.equals(dataBlock.getLongDataName())) { // check for a long name match first
return dataBlock;
}
@ -189,23 +184,19 @@ public class PamConfiguration {
return dataBlock;
}
}
return tryShortDataName(blockType, name);
return tryShortName(blockType, name);
}
/**
* For when the function calling with longdataname fails!
* For backwards compatibility, some blocks may still use the short name.
* @param blockType
* @param name
* @return
*/
private PamDataBlock tryShortDataName(Class blockType, String name) {
private PamDataBlock tryShortName(Class blockType, String name) {
if (name == null) return null;
ArrayList<PamDataBlock> blocks = getDataBlocks(blockType, true);
System.out.println(name);
for (PamDataBlock dataBlock:blocks) {
if (name.equals(dataBlock.getDataName())) { // check for a long name match first
return dataBlock;
}

7
src/PamController/PamController.java
View File

@ -1099,7 +1099,8 @@ public class PamController implements PamControllerInterface, PamSettings {
* later in the AWT event queue.
*/
public void startLater() {
SwingUtilities.invokeLater(new StartLater(true));
// SwingUtilities.invokeLater(new StartLater(true));
startLater(true);
}
public void startLater(boolean saveSettings) {
@ -1173,6 +1174,7 @@ public class PamController implements PamControllerInterface, PamSettings {
@Override
public void manualStop() {
lastStartStopButton = BUTTON_STOP;
setManualStop(true);
pamStop();
}
@ -1186,6 +1188,7 @@ public class PamController implements PamControllerInterface, PamSettings {
@Override
public boolean pamStart() {
// Debug.println("PAMController: pamStart");
setManualStop(false);
return pamStart(true);
}
@ -1434,7 +1437,7 @@ public class PamController implements PamControllerInterface, PamSettings {
pamControlledUnits.get(iU).flushDataBlockBuffers(2000);
}
}
dumpBufferStatus("In pamStopped, now idle", true);
dumpBufferStatus("In pamStopped, now idle", false);
// wait here until the status has changed to Pam_Idle, so that we know
// that we've really finished processing all data

6
src/PamController/PamguardVersionInfo.java
View File

@ -16,7 +16,7 @@ public class PamguardVersionInfo {
* @return release type
*/
static public ReleaseType getReleaseType() {
return ReleaseType.OTHER;
return ReleaseType.BETA;
}
/**
@ -31,12 +31,12 @@ public class PamguardVersionInfo {
* Version number, major version.minorversion.sub-release.
* Note: can't go higher than sub-release 'f'
*/
static public final String version = "2.02.10b";
static public final String version = "2.02.10bd";
/**
* Release date
*/
static public final String date = "2 March 2024";
static public final String date = "23 April 2024";
// /**
// * Release type - Beta or Core

15
src/PamController/command/BatchStatusCommand.java
View File

@ -1,6 +1,10 @@
package PamController.command;
import java.util.ArrayList;
import Acquisition.AcquisitionControl;
import PamController.DataInputStore;
import PamController.PamControlledUnit;
import PamController.PamController;
import offlineProcessing.OfflineTaskManager;
import pamViewFX.PamControlledGUIFX;
@ -35,7 +39,16 @@ public class BatchStatusCommand extends ExtCommand {
}
private String getNormalModeStatus(String command) {
AcquisitionControl daqControl = (AcquisitionControl) PamController.getInstance().findControlledUnit(AcquisitionControl.class, null);
/**
* find a controlled unit thats a DataInputSource which should either be a sound daq or a Tritech daq module.
*/
ArrayList<PamControlledUnit> inputSources = PamController.getInstance().findControlledUnits(DataInputStore.class, true);
if (inputSources.size() == 0) {
return null;
}
// DataInputStore daqControl = (DataInputStore) PamController.getInstance().findControlledUnit(DataInputStore.class, null);
DataInputStore daqControl = (DataInputStore) inputSources.get(0);
// System.out.println("Getting batch status from : " + daqControl);
if (daqControl == null) {
return null;
}

3
src/PamController/settings/output/xml/PamguardXMLWriter.java
View File

@ -570,6 +570,9 @@ public class PamguardXMLWriter implements PamSettings {
*/
private Element writeSettings(Document doc, PamSettings pamSettings, Object data, ArrayList<Object> objectHierarchy) {
if (data == null) {
return null;
}
Element el = doc.createElement("SETTINGS");
el.setAttribute("Type", pamSettings.getUnitType());
el.setAttribute("Name", pamSettings.getUnitName());

8
src/PamUtils/FileList.java
View File

@ -60,17 +60,19 @@ public class FileList {
// System.out.println("Adding files from folder " + folder.getAbsolutePath());
// first go through all the files in this folder
File[] newFiles = folder.listFiles(fileFilter);
if (newFiles == null) {
return fileList; // nothing to do here.
}
for (int i = 0; i < newFiles.length; i++) {
if (!newFiles[i].isDirectory()) {
fileList.add(newFiles[i]);
// System.out.println(" adding file " + newFiles[i].getName());
// System.out.println(" adding file " + newFiles[i].getName());
}
else if (includeSubFolders) {
fileList = addFiles(newFiles[i], fileList, fileFilter, includeSubFolders);
}
}
return fileList;
}
}

12
src/PamUtils/PamUtils.java
View File

@ -354,6 +354,9 @@ public class PamUtils {
* @return output angle (degrees)
*/
static public double constrainedAngle(double angle) {
if (Double.isInfinite(angle) || Double.isNaN(angle)) {
return angle;
}
while (angle >= 360) {
angle -= 360;
}
@ -369,6 +372,9 @@ public class PamUtils {
* @return output angle (radians)
*/
static public double constrainedAngleR(double angle) {
if (Double.isInfinite(angle) || Double.isNaN(angle)) {
return angle;
}
while (angle >= 2*Math.PI) {
angle -= 2*Math.PI;
}
@ -384,6 +390,9 @@ public class PamUtils {
* @return output angle (degrees)
*/
static public double constrainedAngle(double angle, double maxAngle) {
if (Double.isInfinite(angle) || Double.isNaN(angle)) {
return angle;
}
while (angle >= maxAngle) {
angle -= 360;
}
@ -401,6 +410,9 @@ public class PamUtils {
* @return output angle (radians)
*/
static public double constrainedAngleR(double angle, double maxAngle) {
if (Double.isInfinite(angle) || Double.isNaN(angle)) {
return angle;
}
while (angle > maxAngle) {
angle -= 2*Math.PI;
}

19
src/PamUtils/TxtFileUtils.java
View File

@ -6,6 +6,7 @@ import java.io.File;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.math.BigDecimal;
import java.text.DecimalFormat;
import java.text.NumberFormat;
import java.util.ArrayList;
@ -89,10 +90,20 @@ public class TxtFileUtils {
//5/08/2022 - there was a bug here where there is some sort of invisible character that does not appear on the
//print screen - the only way you can tell is the char array is greater than the number of digits - removed all non numeric
//characters.
// updated again on 15/11/23 to include - signs, or you end up with the abs(of every number!)
String number = new String(recordsOnLine[i].strip().replaceAll("[^\\d.-]", ""));
dat = Double.valueOf(number);
//dat = DecimalFormat.getNumberInstance().parse(new String(recordsOnLine[i].strip().toCharArray())).doubleValue();
// if (recordsOnLine[i].contains("e") || recordsOnLine[i].contains("E")) {
// //need to check whether scientific notation might be used here
//// dat = new BigDecimal(recordsOnLine[i]).doubleValue();
// dat = Double.valueOf(recordsOnLine[i]);
// }
// else {
//note that this gets rid of numbers like 3.84e+05
// updated again on 15/11/23 to include - signs, or you end up with the abs(of every number!)
//also added E and e to neglected characters to ensure scientific notation can be used,
String number = new String(recordsOnLine[i].strip().replaceAll("[^Ee\\d.-]", ""));
dat = Double.valueOf(number);
//dat = DecimalFormat.getNumberInstance().parse(new String(recordsOnLine[i].strip().toCharArray())).doubleValue();
// }
}
catch (Exception e){
e.printStackTrace();

17
src/PamUtils/worker/PamWorkDialog.java
View File

@ -1,5 +1,7 @@
package PamUtils.worker;
import java.awt.Dimension;
import java.awt.Point;
import java.awt.Window;
import javax.swing.BoxLayout;
@ -23,6 +25,9 @@ public class PamWorkDialog extends PamDialog {
mainPanel.setBorder(new TitledBorder("Task Progress"));
// GridBagConstraints c = new PamGridBagContraints();
mainPanel.add(progressBar = new JProgressBar(0, 100));
Dimension sz = progressBar.getPreferredSize();
sz.width = 300;
progressBar.setPreferredSize(sz);
textRows = new PamTextDisplay[nTextRows];
for (int i = 0; i < nTextRows; i++) {
// c.gridy++;
@ -35,6 +40,18 @@ public class PamWorkDialog extends PamDialog {
getButtonPanel().setVisible(false);
setDialogComponent(mainPanel);
setResizable(true);
if (parentFrame != null) {
Dimension prefSize = this.getPreferredSize();
Point screenLoc = parentFrame.getLocationOnScreen();
int x = (parentFrame.getWidth()-prefSize.width)/2;
int y = (parentFrame.getHeight()-prefSize.height)/2;
if (screenLoc != null) {
x += screenLoc.x;
y += screenLoc.y;
}
setLocation(x, y);
}
}
public void update(PamWorkProgressMessage progressMsg) {

25
src/PamView/PamGui.java
View File

@ -74,11 +74,8 @@ import javax.swing.event.MenuListener;
import Acquisition.DaqSystemInterface;
import annotation.tasks.AnnotationManager;
import export.ExportOptions;
import metadata.MetaDataContol;
import pamViewFX.fxNodes.pamDialogFX.PamDialogFX2AWT;
import performanceTests.PerformanceDialog;
import rawDeepLearningClassifier.RawDLParams;
import tipOfTheDay.TipOfTheDayManager;
import Array.ArrayManager;
//import Logging.LogDataObserver;
@ -89,7 +86,6 @@ import PamController.PamControllerInterface;
import PamController.PamSettingManager;
import PamController.PamSettings;
import PamController.PamguardVersionInfo;
import PamController.SettingsPane;
import PamController.settings.SettingsImport;
import PamModel.CommonPluginInterface;
import PamModel.PamModel;
@ -672,13 +668,6 @@ public class PamGui extends PamView implements WindowListener, PamSettings {
startMenuEnabler.addMenuItem(menuItem);
fileMenu.add(menuItem);
//export items
menuItem = new JMenuItem("Export Data ...");
menuItem.addActionListener(new ExportOptionsListener(getGuiFrame()));
startMenuEnabler.addMenuItem(menuItem);
fileMenu.add(menuItem);
for (int i = 0; i < pamControllerInterface.getNumControlledUnits(); i++) {
menuItem = pamControllerInterface.getControlledUnit(i).createFileMenu(frame);
@ -986,20 +975,6 @@ public class PamGui extends PamView implements WindowListener, PamSettings {
}
}
class ExportOptionsListener implements ActionListener {
private JFrame parentFrame;
public ExportOptionsListener(JFrame parentFrame) {
super();
this.parentFrame = parentFrame;
}
public void actionPerformed(ActionEvent e) {
//show the export options dialog.
ExportOptions.getInstance().showDialog(parentFrame);
}
}
class MenuGeneralXMLExport implements ActionListener {
private JFrame parentFrame;
public MenuGeneralXMLExport(JFrame parentFrame) {

5
src/PamView/panel/KeyPanel.java
View File

@ -102,7 +102,10 @@ public class KeyPanel {
}
private void fillPanel() {
JPanel panel = this.panel;
if (panel == null) {
return;
}
panel.removeAll();
GridBagConstraints c = new GridBagConstraints();

27
src/PamView/panel/SeparatorBorder.java Normal file
View File

@ -0,0 +1,27 @@
package PamView.panel;
import java.awt.Component;
import java.awt.FontMetrics;
import java.awt.Graphics;
import javax.swing.border.TitledBorder;
public class SeparatorBorder extends TitledBorder {
private static final long serialVersionUID = 1L;
public SeparatorBorder(String title) {
super(title);
// TODO Auto-generated constructor stub
}
public void paintBorder(Component c, Graphics g, int x, int y, int width, int height) {
FontMetrics fm = g.getFontMetrics();
if (fm != null) {
height = fm.getHeight()-EDGE_SPACING*2-1;
}
else {
height = 12;
}
super.paintBorder(c, g, x, y, width, height);
}
}

4
src/PamView/paneloverlay/overlaymark/OverlayMark.java
View File

@ -173,10 +173,10 @@ public class OverlayMark {
//now must find the limits of the zoom. Can be complex shapes so a bit difficult;
double minX=Double.MAX_VALUE;
double maxX=Double.MIN_VALUE;
double maxX=Double.NEGATIVE_INFINITY;
double minY=Double.MAX_VALUE;
double maxY=Double.MIN_VALUE;
double maxY=Double.NEGATIVE_INFINITY;
//now find those maximum, and minimum values,
for (int i=0; i<coordinates.size(); i++){

5
src/PamguardMVC/PamDataBlock.java
View File

@ -636,7 +636,7 @@ public class PamDataBlock<Tunit extends PamDataUnit> extends PamObservable {
/**
* Find a dataunit based on it's database index. If there have been no updates,
* then database indexes should be in order and a fast find canbe used. If
* then database indexes should be in order and a fast find can be used. If
* however, there have been updates, then things will not be in order so it's
* necessary to go through everything from start to end.
*
@ -4296,6 +4296,9 @@ public class PamDataBlock<Tunit extends PamDataUnit> extends PamObservable {
* @param sayEmpties dump info even if a buffer is empty (otherwise, only ones that have stuff still)
*/
public void dumpBufferStatus(String message, boolean sayEmpties) {
if (sayEmpties || unitsAdded > 0) {
System.out.printf("Datablock %s: Added %d data units\n", getLongDataName(), unitsAdded);
}
int nObs = countObservers();
for (int i = 0; i < nObs; i++) {
PamObserver obs = getPamObserver(i);

3
src/PamguardMVC/PamDataUnit.java
View File

@ -564,7 +564,8 @@ abstract public class PamDataUnit<T extends PamDataUnit, U extends PamDataUnit>
/**
* Get the latlong of the mean hydrophone position at the time of
* this detection.
* this detection. If the data unit has a channel bitmap of zero, then
* get the GPS position of the vessel at that time.
* @param recalculate
* @return Lat long of detection origin (usually the position of the reference hydrophone at time of detection)
*/

17
src/PamguardMVC/PamProcess.java
View File

@ -126,6 +126,8 @@ abstract public class PamProcess implements PamObserver, ProcessAnnotator {
private long cpuUsage;
private long lastCPUCheckTime = System.currentTimeMillis();
private double cpuPercent;
private double totalCPU, peakCPU;
private int nCPU;
/**
* Flag for the process to say whether or not it's primary data connection
@ -140,6 +142,11 @@ abstract public class PamProcess implements PamObserver, ProcessAnnotator {
* Last received data unit - used for working out timing offsets.
*/
private PamDataUnit lastAcousticDataunit;
/*
* diagnostic count of all data processed
*/
private int nDataProcessed;
// some flags and variables needed to deal with conversion from
// milliseconds to samples and back
// private boolean acousticDataSource = false;
@ -742,6 +749,7 @@ abstract public class PamProcess implements PamObserver, ProcessAnnotator {
}
// long cpuStart = SystemTiming.getProcessCPUTime();
long threadId = Thread.currentThread().getId();
nDataProcessed++;
long cpuStart = tmxb.getThreadCpuTime(threadId);
newData(o, arg);
if (processCheck != null) {
@ -762,6 +770,12 @@ abstract public class PamProcess implements PamObserver, ProcessAnnotator {
* get percent. Total is -9+3+2 = /!0^4 !
*/
cpuPercent = (double) cpuUsage / (now - lastCPUCheckTime) / 10000.;
// these next two allow us to take an average cpu.
totalCPU += cpuPercent;
nCPU++;
// and hte max cpu (may always go to 100 at end ?)
peakCPU = Math.max(cpuPercent, peakCPU);
lastCPUCheckTime = now;
cpuUsage = 0;
}
@ -1073,6 +1087,9 @@ abstract public class PamProcess implements PamObserver, ProcessAnnotator {
* @param sayEmpties include info even if a buffer is empty.
*/
public void dumpBufferStatus(String message, boolean sayEmpties) {
if (sayEmpties || nDataProcessed > 0) {
System.out.printf("Process %s: ran %d datas, peak CPU %3.1f%%, mean CPU %3.1f%%\n", this.getProcessName(), nDataProcessed, peakCPU, totalCPU/nCPU);
}
ArrayList<PamDataBlock> outputs = getOutputDataBlocks();
try {
for (PamDataBlock output : outputs) {

36
src/PamguardMVC/superdet/SubdetectionInfo.java
View File

@ -63,6 +63,13 @@ public class SubdetectionInfo<T extends PamDataUnit> implements Comparable<Subde
*/
private int childDatabaseIndex;
private int binaryFileIndex;
/**
* Only exists for click data.
*/
private Integer clickNumber;
/**
* @param subDetection2
@ -106,6 +113,7 @@ public class SubdetectionInfo<T extends PamDataUnit> implements Comparable<Subde
binaryFilename = subTableData.getBinaryFilename();
parentID = subTableData.getParentID();
parentUID = subTableData.getParentUID();
clickNumber = subTableData.getClickNumber();
}
public T getSubDetection() {
@ -210,4 +218,32 @@ public class SubdetectionInfo<T extends PamDataUnit> implements Comparable<Subde
this.childDatabaseIndex = childDatabaseIndex;
}
/**
* @return the binaryFileIndex
*/
public int getBinaryFileIndex() {
return binaryFileIndex;
}
/**
* @param binaryFileIndex the binaryFileIndex to set
*/
public void setBinaryFileIndex(int binaryFileIndex) {
this.binaryFileIndex = binaryFileIndex;
}
/**
* @return the clickNumber
*/
public Integer getClickNumber() {
return clickNumber;
}
/**
* @param childUID the childUID to set
*/
public void setChildUID(long childUID) {
this.childUID = childUID;
}
}

47
src/PamguardMVC/superdet/SuperDetDataBlock.java
View File

@ -1,5 +1,6 @@
package PamguardMVC.superdet;
import java.io.File;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.Collections;
@ -15,6 +16,7 @@ import PamguardMVC.PamDataBlock;
import PamguardMVC.PamDataUnit;
import PamguardMVC.PamProcess;
import PamguardMVC.debug.Debug;
import binaryFileStorage.DataUnitFileInformation;
import clickDetector.offlineFuncs.OfflineEventDataUnit;
import generalDatabase.PamSubtableData;
import generalDatabase.SQLLogging;
@ -448,8 +450,8 @@ public class SuperDetDataBlock<Tunit extends SuperDetection, TSubDet extends Pam
int iDone = 0;
while (duIt.hasNext()) {
Tunit aData = duIt.next(); // looping through superdetections.
// if (aData.getDatabaseIndex() == 131) {
// System.out.println("On event 131");
// if (aData.getDatabaseIndex() == 566) {
// System.out.println("On event 566");
// }
if (viewLoadObserver != null) {
viewLoadObserver.sayProgress(1, aData.getTimeMilliseconds(), firstTime, lastTime, iDone++);
@ -514,7 +516,46 @@ public class SuperDetDataBlock<Tunit extends SuperDetection, TSubDet extends Pam
return Long.signum(comp);
}
comp = subTableData.getChildUID() - dataUnit.getUID();
return Long.signum(comp);
if (comp == 0) {
return 0;
}
/**
* Some problems in some datasets where the uid is corrupt. So also check
* on binary file information. This is slower, but since we've already matched on
* time, it doesn't get called very often.
*/
String dbBin = subTableData.getBinaryFilename();
DataUnitFileInformation duFileInfo = dataUnit.getDataUnitFileInformation();
if (dbBin == null || duFileInfo == null) {
return Long.signum(comp);
}
// String duBin = duFileInfo.get
/*
* Note that some bin names are truncated, so need to match on startswith.
*/
File binFile = duFileInfo.getFile();
if (binFile == null) {
return Long.signum(comp);
}
String fName = binFile.getName();
if (fName.startsWith(dbBin) == false) {
return Long.signum(comp);
}
/*
* hope for a non null click number. If there isn't onw, then we'll just have to
* go with the bin file being the same and the millis being the same. It's important
* for clicks though since there can be >1 in a millisecond.
*/
Integer clickNo = subTableData.getClickNumber();
if (clickNo == null) {
return 0;
}
else {
subTableData.setChildUID(dataUnit.getUID());
dataUnit.updateDataUnit(System.currentTimeMillis());
return (int) (clickNo - duFileInfo.getIndexInFile());
}
}
}

1
src/RightWhaleEdgeDetector/species/RWTethysDataProvider.java
View File

@ -29,6 +29,7 @@ public class RWTethysDataProvider extends AutoTethysProvider {
Parameters parameters = detection.getParameters();
parameters.setScore((double) rweDataUnit.rweSound.soundType);
double snr = 20.*Math.log10(rweDataUnit.rweSound.signal/rweDataUnit.rweSound.noise);
snr = AutoTethysProvider.roundDecimalPlaces(snr, 1);
parameters.setSNRDB(snr);
return detection;

10
src/binaryFileStorage/BinaryStore.java
View File

@ -37,6 +37,7 @@ import dataMap.OfflineDataMap;
import dataMap.OfflineDataMapPoint;
import PamController.AWTScheduler;
import PamController.DataInputStore;
import PamController.DataIntegrityChecker;
import PamController.DataOutputStore;
import PamController.OfflineDataStore;
import PamController.PamControlledUnit;
@ -74,6 +75,7 @@ import annotation.binary.AnnotationBinaryData;
import annotation.handler.AnnotationHandler;
import backupmanager.BackupInformation;
import binaryFileStorage.backup.BinaryBackupStream;
import binaryFileStorage.checker.BinaryIntegrityChecker;
import binaryFileStorage.layoutFX.BinaryStoreGUIFX;
/**
@ -867,7 +869,7 @@ PamSettingsSource, DataOutputStore {
return null;
}
int nFiles = fileList.size();
int updateAt = Math.max(nFiles/1000,1);
int updateAt = Math.max(nFiles/100,1);
int fileIndex;
int nNew = 0;
for (int i = 0; i < nFiles; i++) {
@ -946,6 +948,8 @@ PamSettingsSource, DataOutputStore {
PamController.getInstance().notifyModelChanged(PamControllerInterface.CHANGED_OFFLINE_DATASTORE);
// System.out.println("BinaryDataMapMaker really done " + this);
dataMapMaker = null;
getInegrityChecker().checkDataStore();
}
@Override
@ -2593,5 +2597,9 @@ PamSettingsSource, DataOutputStore {
public String getDataLocation() {
return binaryStoreSettings.getStoreLocation();
}
@Override
public DataIntegrityChecker getInegrityChecker() {
return new BinaryIntegrityChecker(this);
}
}

54
src/binaryFileStorage/checker/BinaryIntegrityChecker.java Normal file
View File

@ -0,0 +1,54 @@
package binaryFileStorage.checker;
import java.util.ArrayList;
import PamController.DataIntegrityChecker;
import PamController.PamController;
import PamView.dialog.warn.WarnOnce;
import PamguardMVC.PamDataBlock;
import binaryFileStorage.BinaryStore;
import dataMap.OfflineDataMap;
public class BinaryIntegrityChecker implements DataIntegrityChecker {
private BinaryStore binaryStore;
public BinaryIntegrityChecker(BinaryStore binaryStore) {
this.binaryStore = binaryStore;
}
public boolean checkDataStore() {
checkMapOverlaps();
return true;
}
private boolean checkMapOverlaps() {
boolean ok = true;
ArrayList<PamDataBlock> dataBlocks = binaryStore.getStreamingDataBlocks(false);
for (PamDataBlock aBlock : dataBlocks) {
ok &= checkMapOverlaps(aBlock);
}
return ok;
}
private boolean checkMapOverlaps(PamDataBlock aBlock) {
OfflineDataMap dataMap = aBlock.getOfflineDataMap(binaryStore);
if (dataMap == null) {
return true;
}
long overlaps = dataMap.checkOverlaps();
if (overlaps <= 0) {
return true;
}
String warn = String.format("<html>Binary data %s has overlapping data files, with a maximum overlap of %3.1fs<br>"
+ "This can occur when data have been reprocessed multiple times offline into "
+ "the same folders.<br>Since files sometimes get slightly different names, old files are"
+ "not always overwritten. <br>"
+ "You should determine which files are 'old' by looking at file creation dates and delete them.",
aBlock.getLongDataName(), (double) overlaps / 1000.);
WarnOnce.showNamedWarning("BINOVERLAPWARNING", PamController.getMainFrame(), "Data Integrity Warning", warn, WarnOnce.WARNING_MESSAGE);
return false;
}
}

21
src/clickDetector/dataSelector/ClickDataSelector.java
View File

@ -112,14 +112,23 @@ public class ClickDataSelector extends DataSelector {
// see if there is a super detection and see if it's got a comment.
String comment = oev.getComment();
boolean isAutomatic = false;
if (comment != null) {
if (clickAlarmParameters.onlineAutoEvents && comment.startsWith("Automatic")) {
return true;
}
if (clickAlarmParameters.onlineManualEvents && comment.startsWith("Manual")) {
return true;
}
isAutomatic = comment.startsWith("Automatic");
}
if (isAutomatic && clickAlarmParameters.onlineAutoEvents) {
return true;
}
else if (clickAlarmParameters.onlineManualEvents) {
return true;
}
// if (clickAlarmParameters.onlineAutoEvents && comment.startsWith("Automatic")) {
// return true;
// }
// if (clickAlarmParameters.onlineManualEvents && comment.startsWith("Manual")) {
// return true;
// }
/*
* Otherwise need to work out where the hell the event type is in the
* list of event types and see if it's wanted.

28
src/clickDetector/dataSelector/ClickSelectPanel.java
View File

@ -30,6 +30,8 @@ import PamView.dialog.PamDialog;
import PamView.dialog.PamDialogPanel;
import PamView.dialog.PamGridBagContraints;
import PamView.panel.PamAlignmentPanel;
import PamView.panel.SeparatorBorder;
import PamView.panel.WestAlignedPanel;
public class ClickSelectPanel implements PamDialogPanel {
@ -40,6 +42,8 @@ public class ClickSelectPanel implements PamDialogPanel {
private ClickDataSelector clickDataSelector;
private JPanel mainPanel;
private boolean isViewer;
public static final String mainTip = "You should select options in both the Click Type and the Event Type panels";
public ClickSelectPanel(ClickDataSelector clickDataSelector, boolean allowScores, boolean useEventTypes) {
this.clickDataSelector = clickDataSelector;
@ -79,7 +83,9 @@ public class ClickSelectPanel implements PamDialogPanel {
private LookupList lutList;
public EventTypePanel() {
setBorder(new TitledBorder("Event Type Selection"));
setBorder(new SeparatorBorder("Event Type Selection"));
setToolTipText(mainTip);
}
void setParams() {
@ -102,16 +108,24 @@ public class ClickSelectPanel implements PamDialogPanel {
boxPanel.add(onlineAuto = new JCheckBox("Automatically detected click trains"), c);
onlineAuto.setSelected(clickAlarmParameters.onlineAutoEvents);
useUnassigned.setToolTipText("Clicks that are NOT part of a manual or automatic click train");
onlineManual.setToolTipText("Clicks that are part of a manually marked click train");
onlineAuto.setToolTipText("Clicks that are part of an automatically detected click train");
lutList = LookUpTables.getLookUpTables().getLookupList(ClicksOffline.ClickTypeLookupName);
if (lutList == null) {
return;
}
c.gridy++;
boxPanel.add(new JLabel("OR the following click train types ...", JLabel.LEFT), c);
useType = new JCheckBox[lutList.getList().size()];
for (int i = 0; i < useType.length; i++) {
c.gridy++;
boxPanel.add(useType[i] = new JCheckBox(lutList.getList().get(i).getText()), c);
useType[i].setSelected(clickAlarmParameters.isUseEventType(lutList.getList().get(i).getCode()));
String tip = String.format("Clicks that are part of a click train labelled as %s", lutList.getList().get(i).getText());
useType[i].setToolTipText(tip);
}
}
@ -158,10 +172,9 @@ public class ClickSelectPanel implements PamDialogPanel {
northPanel = new JPanel();
northPanel.setLayout(new GridBagLayout());
GridBagConstraints c = new PamGridBagContraints();
northPanel.setBorder(new TitledBorder("Echoes"));
c.gridwidth = 3;
c.anchor = GridBagConstraints.WEST;
northPanel.add(useEchoes = new JCheckBox("Use Echoes"), c);
northPanel.add(new PamAlignmentPanel(useEchoes = new JCheckBox("Use Echoes"), BorderLayout.WEST), c);
c.gridwidth = 1;
c.gridy++;
c.gridx = 0;
@ -178,11 +191,13 @@ public class ClickSelectPanel implements PamDialogPanel {
northPanel.add(scoreByAmplitude = new JCheckBox("Score by amplitude"), c);
scoreByAmplitude.setVisible(allowScores);
scoreByAmplitude.addActionListener(new AllSpeciesListener());
add(BorderLayout.NORTH, northPanel);
WestAlignedPanel walpn;
add(BorderLayout.NORTH, walpn = new WestAlignedPanel(northPanel));
walpn.setBorder(new SeparatorBorder("Echoes"));
JPanel centralOuterPanel = new JPanel(new BorderLayout());
centralPanel.setLayout(new GridBagLayout());
centralOuterPanel.setBorder(new TitledBorder("Click Type Selection"));
centralOuterPanel.setBorder(new SeparatorBorder("Click Type Selection"));
add(BorderLayout.CENTER, centralOuterPanel);
JScrollPane scrollPane = new DialogScrollPane(new PamAlignmentPanel(centralPanel, BorderLayout.WEST), 10);
@ -197,6 +212,9 @@ public class ClickSelectPanel implements PamDialogPanel {
clearAll.addActionListener(new AutoSelect(false));
centralOuterPanel.add(BorderLayout.SOUTH, new PamAlignmentPanel(centralEastPanel, BorderLayout.WEST));
centralOuterPanel.setToolTipText(mainTip);
setToolTipText(mainTip);
}
void setParams() {

5
src/clickTrainDetector/ClickTrainProcess.java
View File

@ -6,6 +6,7 @@ import PamguardMVC.PamDataBlock;
import PamguardMVC.PamDataUnit;
import PamguardMVC.PamInstantProcess;
import PamguardMVC.PamObservable;
import PamguardMVC.PamProcess;
import clickDetector.ClickDetection;
import clickTrainDetector.layout.CTDataUnitGraphics;
import clickTrainDetector.layout.UnconfirmedCTSymbolManager;
@ -19,7 +20,7 @@ import pamScrollSystem.AbstractScrollManager;
* @author Jamie Macaulay
*
*/
public class ClickTrainProcess extends PamInstantProcess {
public class ClickTrainProcess extends PamProcess {
/**
* The source data block
@ -47,7 +48,7 @@ public class ClickTrainProcess extends PamInstantProcess {
private CTProcessCheck processCheck;
public ClickTrainProcess(ClickTrainControl pamControlledUnit) {
super(pamControlledUnit);
super(pamControlledUnit, null);
this.clickTrainControl=pamControlledUnit;
this.processCheck=new CTProcessCheck(this);

19
src/dataMap/OfflineDataMap.java
View File

@ -844,4 +844,23 @@ abstract public class OfflineDataMap<TmapPoint extends OfflineDataMapPoint> {
obs.updateDataMap(this, mapPoint);
}
}
/**
* Check to see if any data maps overlap in time.
* @return largest overlap between maps. 0 or -ve means no overlap.
*/
public long checkOverlaps() {
long bigLap = Long.MIN_VALUE;
TmapPoint prevPoint = null;
for (TmapPoint mapPoint : mapPoints) {
if (prevPoint != null) {
long olap = prevPoint.getEndTime() - mapPoint.getStartTime();
// if (mapPoint.getStartTime() < prevPoint.getEndTime()) {
bigLap = Math.max(olap, bigLap);
// }
}
prevPoint = mapPoint;
}
return bigLap;
}
}

6
src/generalDatabase/DBControlUnit.java
View File

@ -18,6 +18,7 @@ import generalDatabase.backup.DatabaseBackupStream;
import pamScrollSystem.ViewLoadObserver;
import pamViewFX.pamTask.PamTaskUpdate;
import PamController.AWTScheduler;
import PamController.DataIntegrityChecker;
import PamController.DataOutputStore;
import PamController.OfflineDataStore;
import PamController.PamConfiguration;
@ -524,5 +525,10 @@ public class DBControlUnit extends DBControl implements DataOutputStore {
}
return state;
}
@Override
public DataIntegrityChecker getInegrityChecker() {
// TODO Auto-generated method stub
return null;
}
}

18
src/generalDatabase/PamSubtableData.java
View File

@ -81,6 +81,11 @@ public class PamSubtableData {
* subdetection table, and not the index from the Click_Detector_Clicks table)
*/
private long dbIndex;
/**
* Click number. only exists for clicks table, but needed for corrupt database.
*/
private Integer clickNumber;
/**
*
@ -158,5 +163,18 @@ public class PamSubtableData {
this.childUTC = childUTC;
}
/**
* @return the clickNumber
*/
public Integer getClickNumber() {
return clickNumber;
}
/**
* @param clickNumber the clickNumber to set
*/
public void setClickNumber(Integer clickNumber) {
this.clickNumber = clickNumber;
}
}

6
src/generalDatabase/SQLLogging.java
View File

@ -2077,8 +2077,9 @@ public abstract class SQLLogging {
ArrayList<PamSubtableData> tableList = new ArrayList<PamSubtableData>();
int n = 0;
try {
PamSubtableDefinition subtableTableDef = (PamSubtableDefinition) getTableDefinition();
PamTableItem clickNoItem = subtableTableDef.findTableItem("ClickNo");
while (subtableResults.next()) {
PamSubtableDefinition subtableTableDef = (PamSubtableDefinition) getTableDefinition();
PamTableItem tableItem;
// transferDataFromResult(con.getSqlTypes(), subtableResults);
for (int i = 0; i < subtableTableDef.getTableItemCount(); i++) {
@ -2102,6 +2103,9 @@ public abstract class SQLLogging {
subtableData.setLongName(subtableTableDef.getLongName().getStringValue());
subtableData.setBinaryFilename(subtableTableDef.getBinaryfile().getStringValue());
subtableData.setDbIndex(subtableTableDef.getIndexItem().getIntegerValue());
if (clickNoItem != null) {
subtableData.setClickNumber(clickNoItem.getIntegerValue());
}
try {
subtableData.setChildUID(subtableTableDef.getUidItem().getLongObject());
}

4
src/generalDatabase/lookupTables/LookUpTables.java
View File

@ -141,9 +141,11 @@ public class LookUpTables {
// search for repeats.
for (int i = 0; i < n-1; i++) {
String code = list.get(i).getCode().trim();
String topic1 = list.get(i).getTopic().trim();
for (int j = i+1; j < n; j++) {
String code2 = list.get(j).getCode().trim();
if (code.equals(code2)) {
String topic2 = list.get(j).getTopic().trim();
if (code.equals(code2) && topic1.equals(topic2)) {
isRepeat[j] = true;
nRepeat++;
}

BIN
src/help/JavaHelpSearch/DOCS
View File

Binary file not shown.

BIN
src/help/JavaHelpSearch/DOCS.TAB
View File

Binary file not shown.

BIN
src/help/JavaHelpSearch/OFFSETS
View File

Binary file not shown.

BIN
src/help/JavaHelpSearch/POSITIONS
View File

Binary file not shown.

2
src/help/JavaHelpSearch/SCHEMA
View File

@ -1,2 +1,2 @@
JavaSearch 1.0
TMAP bs=2048 rt=1 fl=-1 id1=6882 id2=1
TMAP bs=2048 rt=1 fl=-1 id1=6905 id2=1

BIN
src/help/JavaHelpSearch/TMAP
View File

Binary file not shown.

2
src/help/Map.jhm
View File

@ -686,6 +686,8 @@
<mapID target="detectors.clickDetectorHelp.docs.ClickDetector_clickSidePanel" url="detectors/clickDetectorHelp/docs/ClickDetector_clickSidePanel.html"/>
<mapID target="classifiers.matchedtemplate.mathchedtemplate" url="classifiers/matchedtemplate/mathchedtemplate.html"/>
<mapID target="localisation.detectiongroup.docs.dglocaliser" url="localisation/detectiongroup/docs/dglocaliser.html"/>
<mapID target="visual_methods.loggerFormsHelp.docs.SHORT" url="visual_methods/loggerFormsHelp/docs/SHORT.html"/>

2
src/help/PAMGUARDHelpProject.xml
View File

@ -6,7 +6,7 @@
<project>
PAMGUARD
</project>
<projectdir>C:\Users\dg50\source\repos\PAMGuardDG_2\src\help</projectdir>
<projectdir>C:\Users\dg50\source\repos\PAMGuardPAMGuard\src\help</projectdir>
<startpage>
index.html
</startpage>

4
src/help/PAMGUARDTOC.xml
View File

@ -557,6 +557,10 @@
<tocitem text="Common Mistakes and Bugs " target="classifiers.rawDeepLearningHelp.docs.rawDeepLearning_Bugs" image="topic"/>
</tocitem>
<tocitem text="Matched Template Classifier ">
<tocitem text="Matched Template Classifier " target="classifiers.matchedtemplate.mathchedtemplate" image="topic"/>
</tocitem>
</tocitem>
<tocitem text="Localisation ">
<tocitem text="Overview " target="localisation.docs.localisation_overview" image="topic"/>

71
src/help/classifiers/matchedtemplate/mathchedtemplate.html Normal file
View File

@ -0,0 +1,71 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html><head>
<link href="../../pamHelpStylesheet.css" type="text/css" rel="STYLESHEET"><title>ROCCA Classifier Overview</title>
</head>
<body>
<h1 id="matched-click-classifier">Matched click classifier</h1>
<h2 id="overview">Overview</h2>
<p>The matched click classifier is an alternative to the in built click classifier in the click detection module which uses two click templates (target and reject) to classify individual clicks detections from the click detection module. The idea behind this classifier is to more accurately classify rarer clicks when there </p>
<h2 id="how-it-works">How it works</h2>
<p>The classifier is based on a matched filter i.e. a candidate click detection is compared to a template and it&#39;s maximum correlation value with the template is then used for classification. Each click is compared to both a <em>match</em> and a <em>reject</em> template. If the difference between the correlation match to the reject template exceeds a certain threshold, then the click is classified. There can be multiple combinations of match to reject templates allowing the matched classifier to deal with different types of clicks or even clicks from multiple species.</p>
<h2 id="configuring-the-matched-click-classifier">Configuring the matched click classifier</h2>
<p>The matched click classifier settings are accessed via <strong>Settings-&gt; Matched click classifier</strong>_. The settings are split into the three sections, general settings, click waveform and click templates. </p>
<p align="center">
<img width="950" height="520" src = "resources/matched_click_dialog_summary.png">
</p>
<p><em>The settings pane of the matched click classifier</em></p>
<h3 id="general-settings">General Settings</h3>
<p>The general settings allow for channel options, species ID and the default colours for classified clicks to be to be set.
_
Channel Options _allows users to define whether a single click from one channel, all clicks or an averaged click should be used to be used for classification in multi-channel situations. if there is only one channel then this makes no difference. </p>
<p><em>Click Type</em> sets the number that defines the species ID. Make sure this is not the same as any of the species IDs in the default click classifier (this is why the default is set so high). </p>
<p><em>Symbol</em> and <em>Fill</em> define the default colours clicks which have been classified by the matched click classifier should be plotted on displays. </p>
<h3 id="click-waveform-settings">Click Waveform Settings</h3>
<p>Before a click is classified it undergoes some pre-conditioning by the matched click classifier. </p>
<p><em>Restrict parameter extraction to XX samples</em> sets the maximum length of the waveform to the classifier. If this is selected, then center of the click is located and samples trimmed around the center. In some use cases, for example SoundTrap detections, which may be 10,000 samples long, setting a lower number of maximum samples can greatly increase processing speed and improve the accuracy of results. Note that if the number of samples is set too low (e.g. well below the click template length) then the correaltion values will not make much sense.</p>
<p><em>Peak threshold</em> and <em>Smoothing</em> are both parameters used to find the click center so that a click can be trimmed to the maximum number of samples. The click length is measured by calculating the waveform envelope using a Hilbert Transform. The envelope is smoothed using a moving average filter (the <em>Smoothing</em> parameter defines the size of the averaging window). The click is trimmed as follows. First the peak of the waveform envelope is found. The length of the click is defined as the point at which the click falls <em>Peak threshold</em> dB below the peak. The center of the click is then the middle of this snippet. The click is then trimmed from the center of the click. </p>
<p><em>Amplitude normalisation</em> If there is a very loud click compared to a template it&#39;s correlation score will be different to that of a very quiet click of with exactly the same waveform. It is therefore a good idea to normalise the waveform before it is compared the match click classifier. The types of normalisation are </p>
<ul>
<li><em>norm</em> - this is the default - the click is divided by it&#39;s RMS amplitude. </li>
<li><em>peak to peak</em> this can be useful for some types of shorter click e.g. dolphins - the click is divided by it&#39;s peak to peak amplitude.</li>
<li><em>none</em> no normalisation (not recommended).</li>
</ul>
<h3 id="template-settings">Template settings</h3>
<p>The matched click classifier has two templates, a match and a reject. The match template should match a stereotypical click of the target species and the reject template can either be a click from a confounding species (e.g. dolphin) or blank. Selecting the drop-down menu from the <em>Import</em> button allows for a selection of some default templates. Custom templates can be imported by selecting the <em>Import</em> button. Custom templates can either be a .csv. or .mat file using the following format. </p>
<p><strong>.csv</strong> - the first <em>row</em> are the waveform measurements from -1 to 1 (make sure you save with a sufficient number of decimal points). The second row and first column is the sample rate in sample per second.</p>
<p><strong>.mat</strong> - a .mat file that contains two variables named <em>sR</em> and <em>waveform</em>. <em>sR</em> is the sample rate in samples per second and <em>waveform</em> is a 1D array containing the waveform for the template form -1 to 1. Arrays of click structures imported using the PAMGuard to MATLAB library can also be used. The first channel of the first click in a list of clicks structures will be imported as the template. The click structure should be names <em>clicks</em> and the sample rate should be saved as a seperate variable named <em>sR</em> in sampes per second.</p>
<p>The match and reject templates are plotted to provide a user with some visualisation of the classifier settings - the drop down menu allows the user to select different ways to plot the templates and is purely for visualisation purposes so makes no difference to classifier settings. </p>
<p><em>threshold</em> is the threshold at which a click is classified. If the difference between the match and reject templates is above the <em>threshold</em> value then the
click is classified. </p>
<p>The + button can be used to add more tabs. Each tab contains a click/reject template pair and unique threshold setting. A click is classified if at least one of the match/reject templates passes it&#39;s set threshold. </p>
<h2 id="visualising-results">Visualising Results</h2>
<p>The matched click classifier changes the species type flag of a click if at least one of the classifiers passes threshold. This means clicks can be visualised in the click bearing time display or the time base display. The classifier also saves the correlation values for each match/reject template pair which can be accessed in MATLAB/R or through the PAMGuard GUI. </p>
<h3 id="click-bearing-time-display">Click bearing time display</h3>
<p>Matched clicks can be viewed in the bearing time display. If a click passes the threshold of one match/reject pair then the click symbol (defined in general settings) is shown in the bearing time display if <em>Colour by Matched Template Classifier</em> is selected in the right click menu. The correlation values are shown by hovering the mouse over a click to bring up the info tool tip. </p>
<p align="center">
<img width="950" height="550" src = "resources/matched_click_bt_display.png">
</p>
<p><em>Screenshot of clicks classified from the matched click classifier showing matched clicks (coloured pink)</em></p>
<h3 id="time-base-display">Time base display</h3>
<p>The time base display FX can show clicks classified by the matched clicks classifier. <em>Colour by Matched Template Classifier</em> must be selecting in the right settings window. </p>
<p align="center">
<img width="950" height="520" src = "resources/matched_click_tdisplay_example.png">
</p>
<p><em>Screenshot of Time Base display FX showing clicks classified by the match click classifier (coloured pink). The correlation values can be found in the meta data section of the detection pop up menu (highlighted)</em></p>
<p>Note that the time base display allows users to export clicks to be used as templates. Using the advanced pop up menu right click on a click detection and select the MATLAB icon. A .mat file of the selected click or clicks will be saved to your root user folder in a folder called <em>PAMGuard Manual Export</em>. This .mat file can be opened by the matched click classifier - the first channel of the first click in the list will be imported as a template. </p>
<p align="center">
<img width="510" height="300" src = "resources/exporting_mat_clicks.png">
</p>
<h3 id="extracting-correlation-values-using-matlab-r">Extracting correlation values using MATLAB/R</h3>
<p>Clicks are loaded from binary files using the MATLAB/R function </p>
<pre><code><span class="hljs-attribute">clicks</span> = loadPamguardBinaryFile(<span class="hljs-string">"/path/to/pamguardfile.pgdf"</span>)<span class="hljs-comment">;</span>
</code></pre><p>where clicks is a list of MATLAB/R structures containing the data for each click in the file. </p>
<p>The matched click classifier template threshold, match value and reject value are then accessed from each click using</p>
<pre><code><span class="hljs-attr">matchedtemplatevals</span> = clicks(<span class="hljs-number">1</span>).annotations. mclassification
</code></pre><p><code>matachedtemplatevals</code> is a list of where each row is the threshold value, match value and reject value for each match/reject template pair. </p>

BIN
src/help/classifiers/matchedtemplate/resources/exporting_mat_clicks.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 164 KiB

BIN
src/help/classifiers/matchedtemplate/resources/matched_click_bt_display.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.4 MiB

BIN
src/help/classifiers/matchedtemplate/resources/matched_click_dialog_summary.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 702 KiB

BIN
src/help/classifiers/matchedtemplate/resources/matched_click_tdisplay_example.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 629 KiB

15
src/metadata/MetaDataContol.java
View File

@ -7,13 +7,11 @@ import java.io.Serializable;
import javax.swing.JFrame;
import javax.swing.JMenuItem;
import Array.ArrayManager;
import PamController.PamControlledUnit;
import PamController.PamControlledUnitSettings;
import PamController.PamController;
import PamController.PamSettingManager;
import PamController.PamSettings;
import PamModel.PamModuleInfo;
import metadata.swing.MetaDataDialog;
/**
@ -46,25 +44,12 @@ public class MetaDataContol extends PamControlledUnit implements PamSettings {
public static MetaDataContol getMetaDataControl() {
if (singleInstance == null) {
singleInstance = new MetaDataContol(unitType);
singleInstance.addModuleInfo(); //needed for FX
// add this line to add it to the main modules list. Then it will get menu's, etc.
PamController.getInstance().addControlledUnit(singleInstance);
}
return singleInstance;
}
/**
* Add module info to the array manager. Need to do this to add icon which is used in data model.
*/
private void addModuleInfo(){
//need to add module info due to fact array manager is a special case
PamModuleInfo arrayModuleInfo=new PamModuleInfo("MetaDataControl", "Meta Data MANAGER", MetaDataContol.class);
arrayModuleInfo.setCoreModule(true);
arrayModuleInfo.setToolTipText("Meta data manager");
this.setPamModuleInfo(arrayModuleInfo);
}
/**
* Get PAMGuard Metadata. This contains a nilus Deployment object wrapped up
* so that it can be serialised into other PAMGuard settings.

7
src/pamScrollSystem/PamScrollSlider.java
View File

@ -39,6 +39,7 @@ public class PamScrollSlider extends AbstractPamScrollerAWT {
panel.add(BorderLayout.CENTER, slider);
panel.add(BorderLayout.SOUTH, getButtonPanel());
}
// slider.setFocusable(false);
}
class SliderListener implements ChangeListener {
@ -180,5 +181,11 @@ public class PamScrollSlider extends AbstractPamScrollerAWT {
// TODO Auto-generated method stub
super.setPageStep(pageStep);
}
/**
* @return the slider
*/
public JSlider getSlider() {
return slider;
}
}

5
src/serialComms/SerialPortPanel.java
View File

@ -236,9 +236,12 @@ public class SerialPortPanel extends Object {
}
}
public String getPort() {
if (portList == null) return null;
Object selItem = portList.getSelectedItem();
if (selItem == null) {
return null;
}
return portList.getSelectedItem().toString();
}

38
src/tethys/CollectionHandler.java Normal file
View File

@ -0,0 +1,38 @@
package tethys;
abstract public class CollectionHandler {
private Collection collection;
protected TethysControl tethysControl;
/**
* @param tethysControl
* @param collection
*/
public CollectionHandler(TethysControl tethysControl, Collection collection) {
this.tethysControl = tethysControl;
this.collection = collection;
}
public String collectionName() {
return collection.collectionName();
}
/**
* @return the collection
*/
public Collection getCollection() {
return collection;
}
/**
* @return the tethysControl
*/
public TethysControl getTethysControl() {
return tethysControl;
}
public abstract String getHelpPoint();
}

45
src/tethys/TethysControl.java
View File

@ -32,6 +32,8 @@ import PamController.PamSettingManager;
import PamController.PamSettings;
import PamUtils.PamFileChooser;
import PamUtils.PamFileFilter;
import PamUtils.worker.PamWorkWrapper;
import PamUtils.worker.PamWorker;
import PamView.PamTabPanel;
import PamView.dialog.warn.WarnOnce;
import PamguardMVC.PamDataBlock;
@ -104,7 +106,9 @@ public class TethysControl extends PamControlledUnit implements PamSettings, Tet
serverCheckTimer = new Timer(10000, new ActionListener() {
@Override
public void actionPerformed(ActionEvent e) {
checkServer();
ServerStatus serverState = checkServer();
// check less often when it's OK to generate less debug output.
serverCheckTimer.setDelay(serverState.ok ? 600000 : 5000);
}
});
serverCheckTimer.setInitialDelay(0);
@ -469,6 +473,8 @@ public class TethysControl extends PamControlledUnit implements PamSettings, Tet
// case PamControllerInterface.INITIALIZATION_COMPLETE:
initializationStuff();
break;
case PamControllerInterface.HYDROPHONE_ARRAY_CHANGED:
sendStateUpdate(new TethysState(StateType.UPDATEMETADATA));
}
}
@ -525,7 +531,32 @@ public class TethysControl extends PamControlledUnit implements PamSettings, Tet
}
}
private void countProjectDetections() {
// /**
// * Count project detections in a worker thread;
// */
// public void countProjectDetectionsT() {
// sendStateUpdate(new TethysState(StateType.EXPORTRDATA, Collection.Detections));
// PamWorker<String> worker = new PamWorker<String>(new PamWorkWrapper<String>() {
//
// @Override
// public String runBackgroundTask(PamWorker<String> pamWorker) {
// countProjectDetections();
// return null;
// }
//
// @Override
// public void taskFinished(String result) {
// sendStateUpdate(new TethysState(StateType.NEWPAMGUARDSELECTION, Collection.Detections));
//
// }
// }, getGuiFrame(), 0, "Updating detections counts");
// worker.start();
// }
/**
* Count project detections in the current thread.
*/
private synchronized void countProjectDetections() {
if (dataBlockSynchInfos == null) {
return;
}
@ -549,12 +580,6 @@ public class TethysControl extends PamControlledUnit implements PamSettings, Tet
i++;
}
// int[] counts = dbxmlQueries.countDataForProject(deplData.getProject(), dataPrefixes);
// if (counts != null) {
// for ( i = 0; i < counts.length; i++ ) {
// dataBlockSynchInfos.get(i).setDataCount(counts[i]);
// }
// }
}
/**
@ -688,11 +713,9 @@ public class TethysControl extends PamControlledUnit implements PamSettings, Tet
* @param dataBlock
*/
public void exportedDetections(PamDataBlock dataBlock) {
sendStateUpdate(new TethysState(StateType.EXPORTRDATA, Collection.Detections));
countProjectDetections();
sendStateUpdate(new TethysState(StateType.NEWPAMGUARDSELECTION, Collection.Detections));
// sendStateUpdate(new TethysState(StateType.NEWPAMGUARDSELECTION, Collection.Detections));
}
/**
* @return the calibrationHandler
*/

13
src/tethys/TethysTimeFuncs.java
View File

@ -12,6 +12,7 @@ import javax.xml.datatype.DatatypeFactory;
import javax.xml.datatype.XMLGregorianCalendar;
import PamUtils.PamCalendar;
import nilus.Helper;
public class TethysTimeFuncs {
@ -54,6 +55,18 @@ public class TethysTimeFuncs {
* @return
*/
public static XMLGregorianCalendar fromGregorianXML(String gregorianString) {
try {
XMLGregorianCalendar xmlCal = Helper.timestamp(gregorianString);
return xmlCal;
} catch (DatatypeConfigurationException e1) {
// TODO Auto-generated catch block
// e1.printStackTrace();
}
/**
* Above should work just fine. If it doesn't use my own code below...
*/
// typical string is 2018-10-20T00:00:00Z
if (gregorianString == null) {
return null;

17
src/tethys/calibration/CalibrationHandler.java
View File

@ -1,6 +1,5 @@
package tethys.calibration;
import java.lang.reflect.Field;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
@ -37,6 +36,7 @@ import nilus.MetadataInfo;
import nilus.QualityValueBasic;
import nilus.ResponsibleParty;
import tethys.Collection;
import tethys.CollectionHandler;
import tethys.DocumentInfo;
import tethys.DocumentNilusObject;
import tethys.TethysControl;
@ -52,9 +52,7 @@ import tethys.niluswraps.NilusUnpacker;
import tethys.pamdata.AutoTethysProvider;
import tethys.reporter.TethysReporter;
public class CalibrationHandler implements TethysStateObserver {
private TethysControl tethysControl;
public class CalibrationHandler extends CollectionHandler implements TethysStateObserver {
private ArrayList<DocumentNilusObject<Calibration>> calibrationsList;
@ -65,10 +63,14 @@ public class CalibrationHandler implements TethysStateObserver {
public static final String[] qaTypes = {"unverified", "valid", "invalid"};
private Helper nilusHelper;
public static final String helpPoint = "utilities.tethys.docs.calibrations";
/**
* @param tethysControl
*/
public CalibrationHandler(TethysControl tethysControl) {
super(tethysControl, Collection.Calibrations);
this.tethysControl = tethysControl;
calibrationsList = new ArrayList();
tethysControl.addStateObserver(this); try {
@ -504,7 +506,7 @@ public class CalibrationHandler implements TethysStateObserver {
}
String seachPattern = makeChannelNamePart(iChan);
for (int i = 0; i < calibrationsList.size(); i++) {
String docName = calibrationsList.get(i).getDocumentName();
String docName = calibrationsList.get(i).getDocumentId();
if (docName.endsWith(seachPattern)) {
return true;
}
@ -570,4 +572,9 @@ public class CalibrationHandler implements TethysStateObserver {
}
return theseCals;
}
@Override
public String getHelpPoint() {
return helpPoint;
}
}

33
src/tethys/calibration/swing/CalibrationsContactCard.java
View File

@ -5,6 +5,7 @@ import java.awt.GridBagConstraints;
import java.awt.GridBagLayout;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.util.ArrayList;
import java.util.Date;
import javax.swing.BoxLayout;
@ -18,6 +19,9 @@ import javax.xml.datatype.XMLGregorianCalendar;
import org.jdesktop.swingx.JXDatePicker;
import Array.ArrayManager;
import Array.Hydrophone;
import Array.PamArray;
import PamView.dialog.PamDialog;
import PamView.dialog.PamGridBagContraints;
import PamView.panel.WestAlignedPanel;
@ -43,6 +47,8 @@ public class CalibrationsContactCard extends CalibrationsCard {
private JButton copyDown, copyUp;
private JComboBox<String> hydrophoneSelection;
public CalibrationsContactCard(PamWizard pamWizard) {
super(pamWizard, "Contact Details");
// TODO Auto-generated constructor stub
@ -57,17 +63,26 @@ public class CalibrationsContactCard extends CalibrationsCard {
JPanel datePanel = new JPanel(new GridBagLayout());
JPanel lp = new WestAlignedPanel(datePanel);
lp.setBorder(new TitledBorder("Calibration date"));
lp.setBorder(new TitledBorder("Date and hydrophones"));
GridBagConstraints c = new PamGridBagContraints();
datePanel.add(new JLabel("Calibration date: ", JLabel.RIGHT), c);
datePicker = new JXDatePicker();
c.gridx++;
datePanel.add(datePicker, c);
c.gridx = 0;
c.gridy++;
datePanel.add(new JLabel("Update Frequency", JLabel.RIGHT), c);
// c.gridx = 0;
c.gridx++;
datePanel.add(new JLabel(" Update Frequency ", JLabel.RIGHT), c);
c.gridx++;
datePanel.add(updateInterval, c);
c.gridx = 0;
c.gridy++;
datePanel.add(new JLabel(" Hydrophones ", JLabel.RIGHT), c);
c.gridwidth = 5;
c.gridx++;
hydrophoneSelection = new JComboBox<>();
datePanel.add(hydrophoneSelection, c);
hydrophoneSelection.setToolTipText("Select which hydrophone calibrations to export");
calibrator = new ResponsiblePartyPanel("Technical Person");
dataManager = new ResponsiblePartyPanel("Data Manager");
@ -202,6 +217,16 @@ public class CalibrationsContactCard extends CalibrationsCard {
datePicker.setDate(new Date(TethysTimeFuncs.millisFromGregorianXML(ts)));
}
hydrophoneSelection.removeAllItems();
hydrophoneSelection.addItem("All hydrophones");
PamArray array = ArrayManager.getArrayManager().getCurrentArray();
ArrayList<Hydrophone> phones = array.getHydrophoneArray();
int i = 0;
for (Hydrophone phone : phones) {
String txt = String.format("Hydrophone %d, %s, %3.1f dBre1\u00B5Pa", i, phone.getType(), phone.getSensitivity());
hydrophoneSelection.addItem(txt);
i++;
}
}

4
src/tethys/calibration/swing/CalibrationsExportWizard.java
View File

@ -5,6 +5,7 @@ import java.awt.Window;
import PamView.wizard.PamWizard;
import PamView.wizard.PamWizardCard;
import nilus.Calibration;
import tethys.calibration.CalibrationHandler;
public class CalibrationsExportWizard extends PamWizard {
@ -13,8 +14,9 @@ public class CalibrationsExportWizard extends PamWizard {
private CalibrationsExportWizard(Window parentFrame, Calibration sampleDocument) {
super(parentFrame, "Calibrations Export");
this.sampleDocument = sampleDocument;
addCard(new CalibrationProcessCard(this));
addCard(new CalibrationsContactCard(this));
addCard(new CalibrationProcessCard(this));
setHelpPoint(CalibrationHandler.helpPoint);
}
public static Calibration showWizard(Window parentFrame, Calibration sampleDocument) {

98
src/tethys/calibration/swing/CalibrationsMainPanel.java
View File

@ -15,53 +15,47 @@ import PamView.panel.PamPanel;
import tethys.TethysControl;
import tethys.TethysState;
import tethys.calibration.CalibrationHandler;
import tethys.deployment.PInstrument;
import tethys.swing.TethysExportPanel;
import tethys.swing.TethysGUIPanel;
import tethys.swing.TippedButton;
public class CalibrationsMainPanel extends TethysGUIPanel {
public class CalibrationsMainPanel extends TethysExportPanel {
private CalibrationHandler calibrationHandler;
private CalibrationsTable calibrationsTable;
private JPanel mainPanel;
private JPanel ctrlPanel;
private JButton exportButton;
private JLabel warning;
// private JPanel mainPanel;
//
// private JPanel ctrlPanel;
//
// private TippedButton exportButton;
//
// private JLabel warning;
public CalibrationsMainPanel(TethysControl tethysControl, CalibrationHandler calibrationHandler) {
super(tethysControl);
super(tethysControl, calibrationHandler, false);
this.calibrationHandler = calibrationHandler;
mainPanel = new PamPanel(new BorderLayout());
JPanel mainPanel = getMainPanel();
// mainPanel = new PamPanel(new BorderLayout());
mainPanel.setBorder(new TitledBorder("Instrument calibration information"));
calibrationsTable = new CalibrationsTable(tethysControl, calibrationHandler);
mainPanel.add(BorderLayout.CENTER, calibrationsTable.getComponent());
ctrlPanel = new PamPanel(new BorderLayout());
exportButton = new JButton("Export ...");
ctrlPanel.add(BorderLayout.WEST, exportButton);
warning = new JLabel();
ctrlPanel.add(BorderLayout.CENTER, warning);
mainPanel.add(BorderLayout.NORTH, ctrlPanel);
exportButton.setToolTipText("Export calibration data to database");
exportButton.addActionListener(new ActionListener() {
@Override
public void actionPerformed(ActionEvent e) {
exportCalibrations();
}
});
}
protected void exportCalibrations() {
calibrationHandler.exportAllCalibrations();
}
@Override
public JComponent getComponent() {
return mainPanel;
// ctrlPanel = new PamPanel(new BorderLayout());
// exportButton = new TippedButton("Export ...", "Export calibration data to database");
// ctrlPanel.add(BorderLayout.WEST, exportButton);
// warning = new JLabel();
// ctrlPanel.add(BorderLayout.CENTER, warning);
// mainPanel.add(BorderLayout.NORTH, ctrlPanel);
// exportButton.addActionListener(new ActionListener() {
// @Override
// public void actionPerformed(ActionEvent e) {
// exportCalibrations();
// }
// });
}
@Override
@ -71,7 +65,45 @@ public class CalibrationsMainPanel extends TethysGUIPanel {
}
private void enableControls() {
exportButton.setEnabled(getTethysControl().isServerOk());
if (getTethysControl().isServerOk() == false) {
disableExport("Tethys Server not running");
return;
}
if (isHydrophoneNamed() == false) {
disableExport("Can't export calibrations until the Hydrophone array has been correctly named");
return;
};
enableExport(true);
}
/**
* Check to see if hydrophone is named correctly.
* @return
*/
private boolean isHydrophoneNamed() {
PInstrument currentInstrument = getTethysControl().getDeploymentHandler().getCurrentArrayInstrument();
if (currentInstrument == null) {
return false;
}
if (currentInstrument.instrumentId == null || currentInstrument.instrumentType == null) {
return false;
}
if (currentInstrument.instrumentId.length() == 0 || currentInstrument.instrumentType.length() == 0) {
return false;
}
return true;
}
@Override
protected void exportButtonPressed(ActionEvent e) {
calibrationHandler.exportAllCalibrations();
}
@Override
protected void optionsButtonPressed(ActionEvent e) {
// TODO Auto-generated method stub
}
}

1
src/tethys/calibration/swing/CalibrationsTable.java
View File

@ -120,6 +120,7 @@ public class CalibrationsTable extends TethysGUIPanel {
});
popMenu.add(menuItem);
}
popMenu.addSeparator();
if (n > 1) {
menuItem = new JMenuItem("Delete selected documents");
menuItem.addActionListener(new ActionListener() {

120
src/tethys/dbxml/DBXMLConnect.java
View File

@ -3,14 +3,17 @@ package tethys.dbxml;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStream;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.zip.GZIPOutputStream;
import javax.xml.bind.JAXBException;
@ -67,7 +70,7 @@ public class DBXMLConnect {
}
return true;
}
/**
* Get the client. The client will only be recreated if the url changes
@ -113,9 +116,9 @@ public class DBXMLConnect {
}
return file;
}
/**
* Create a temporary nilus file.
* Create a temporary nilus file.
* @param nilusObject
* @return
* @throws TethysException
@ -124,20 +127,20 @@ public class DBXMLConnect {
String tempName = getTempFileName(nilusObject);
tempName = tempDirectory.getAbsolutePath() + File.separator + tempName + ".xml";
File tempFile = new File(tempName);
File retFile = createXMLDocument(nilusObject, tempFile);
File retFile = createXMLDocument(nilusObject, tempFile);
retFile.deleteOnExit();
return retFile;
}
public boolean postAndLog(Object nilusObject) throws TethysException
{
return postAndLog(nilusObject, null);
}
/**
* I don't think this should ever be used since everything goes a bit pear
* shaped if the documentName isn't the same as the Id. However, for Calibration
* shaped if the documentName isn't the same as the Id. However, for Calibration
* documents this is no longer the case, since a Calibration can have multiple
* entries on different dates, so allow it !
* @param nilusObject
@ -145,11 +148,11 @@ public class DBXMLConnect {
* @return
* @throws TethysException
*/
public boolean postAndLog(Object nilusObject, String documentName) throws TethysException
{
public boolean postAndLog(Object nilusObject, String documentName) throws TethysException
{
boolean ok = NilusChecker.warnEmptyFields(tethysControl.getGuiFrame(), nilusObject);
TethysException e = null;
boolean success = false;
try {
@ -168,13 +171,13 @@ public class DBXMLConnect {
}
return success;
}
/**
* take a nilus object loaded with PamGuard data and post it to the Tethys database
*
* @param pamGuardObjs a nilus object loaded with PamGuard data
* @return error string, null string means there are no errors
* @throws TethysException
* @throws TethysException
*/
private boolean postToTethys(Object nilusObject, String documentName) throws TethysException
{
@ -192,12 +195,35 @@ public class DBXMLConnect {
MarshalXML marshal = new MarshalXML();
marshal.createInstance(objClass);
marshal.marshal(nilusObject, tempFile.toString());
// above lines have made a file. Are now going to gzip it before sending to Tethys
File zipFile = null;
try {
zipFile = zipOutputFile(tempFile);
}
catch (FileNotFoundException e1){
System.out.println(e1.getMessage());
}
catch (IOException e2) {
System.out.println(e2.getMessage());
}
String finalName;
if (zipFile == null) {
finalName = bodgeName;
}
else {
finalName = zipFile.toString();
}
// tempFile = stripXMLHeader(tempFile);
importReturn = Importer.ImportFiles(params.getFullServerName(), collection.collectionName(),
new String[] { bodgeName }, "", "", false);
new String[] { finalName }, "", "", false);
tempFile.deleteOnExit();
if (zipFile != null) {
zipFile.deleteOnExit();
}
} catch(IllegalArgumentException e) {
throw new TethysException("IllegalArgumentException posting to Tethys: " + e.getMessage(), null);
} catch (IOException e) {
@ -207,21 +233,49 @@ public class DBXMLConnect {
}
/*
* The returned string consists of the file name, then an XML report.
* Quite hard to see much common structure in this, so just look for
* The returned string consists of the file name, then an XML report.
* Quite hard to see much common structure in this, so just look for
* two words, <Success> and <Error>
*/
boolean error = importReturn.contains("<Error");
boolean success = importReturn.contains("<Success>");
String name = tempFile.getName();
TethysReporter.getTethysReporter().addReport(new TethysReport(success, collection, name, name));
// error = !success; might be a better options.
// error = !success; might be a better options.
if (error) {
throw new TethysException("Error posting to Tethys", importReturn);
}
return success;
}
/**
* Zip an xml file (or any file) into a gz file with a new end
* @param xmlFile
* @return
* @throws FileNotFoundException
* @throws IOException
*/
private File zipOutputFile(File xmlFile) throws FileNotFoundException, IOException {
String zipName = xmlFile.toString() + "-temp-.gz";
File zipFile = new File(zipName);
GZIPOutputStream opStream = new GZIPOutputStream(new FileOutputStream(zipFile));
InputStream fis = new FileInputStream(xmlFile);
int chunkSize = 100*1024;
byte[] buffer = new byte[chunkSize];
// ZipEntry zipEntry = new ZipEntry(xmlFile.getName());
int bytesRead;
while ((bytesRead = fis.read(buffer)) >= 0) {
opStream.write(buffer, 0, bytesRead);
}
opStream.close();
fis.close();
return zipFile;
}
/**
* Update a document within Tethys. We're assuming that a
* document with the same name in the same collection already
@ -229,7 +283,7 @@ public class DBXMLConnect {
* the removedocument function
* @param nilusDocument
* @return
* @throws TethysException
* @throws TethysException
*/
public boolean updateDocument(Object nilusDocument) throws TethysException {
deleteDocument(nilusDocument);
@ -242,7 +296,7 @@ public class DBXMLConnect {
* class to identify the correct collection.
* @param nilusDocument
* @return
* @throws TethysException
* @throws TethysException
*/
public boolean deleteDocument(Object nilusDocument) throws TethysException {
@ -259,7 +313,7 @@ public class DBXMLConnect {
<DELETE>
<ITEM> ['ECoastNARW0'] </ITEM>
</DELETE>
An error will throw an exception.
An error will throw an exception.
*/
}
catch (Exception e) {
@ -275,7 +329,7 @@ An error will throw an exception.
* Delete a Deploymnet and any contained Detections document. Doesn't work !
* @param deploymentId
* @return
* @throws TethysException
* @throws TethysException
*/
public boolean deleteDeployment(String deploymentId) throws TethysException {
ArrayList<String> detDocNames = tethysControl.getDbxmlQueries().getDetectionsDocuments(deploymentId);
@ -292,8 +346,8 @@ An error will throw an exception.
}
/**
* Remove a document based on a collection name and a cdocument Id.
* @param collection collection name.
* Remove a document based on a collection name and a cdocument Id.
* @param collection collection name.
* @param documentName document name (not the internal Document Id)
* @return
* @throws TethysException
@ -303,8 +357,8 @@ An error will throw an exception.
}
/**
* Remove a document based on a collection name and a document namw.
* @param collectionName collection name.
* Remove a document based on a collection name and a document namw.
* @param collectionName collection name.
* @param documentName document name (not the internal Document Id)
* @return
* @throws TethysException
@ -319,7 +373,7 @@ An error will throw an exception.
<DELETE>
<ITEM> ['ECoastNARW0'] </ITEM>
</DELETE>
An error will throw an exception.
An error will throw an exception.
*/
}
catch (Exception e) {
@ -330,14 +384,14 @@ An error will throw an exception.
}
/**
* check the return string from importFiles and if it's an
* error, throw an exception. Otherwise do nothing.
* check the return string from importFiles and if it's an
* error, throw an exception. Otherwise do nothing.
* @param fileError
*/
private void checkReturnString(String fileError) {
/**
* Example good string is
*
*
C:\Users\dg50\AppData\Local\Temp\PAMGuardTethys\20080311_2DSimplex_0.xml: 7360 bytes
<?xml version="1.0" encoding="iso-8859-1"?>
<Import>
@ -367,7 +421,7 @@ C:\Users\dg50\AppData\Local\Temp\PAMGuardTethys\20080311_2DSimplex_0.xmlnot: 0 b
/**
* Seems we have to get rid of the line <?xml version="1.0" encoding="UTF-8"?>
* which is being put there by the marshaller ?
* which is being put there by the marshaller ?
* @param tempFile
*/
private File stripXMLHeader(File tempFile) {
@ -377,7 +431,7 @@ C:\Users\dg50\AppData\Local\Temp\PAMGuardTethys\20080311_2DSimplex_0.xmlnot: 0 b
try {
BufferedReader reader = new BufferedReader(new FileReader(tempFile));
BufferedWriter writer = new BufferedWriter(new FileWriter(tempTemp));
String line = reader.readLine();
String line = reader.readLine();
while (line != null) {
// see if the line has any unicode in it
int len = line.length();

3
src/tethys/dbxml/DBXMLQueries.java
View File

@ -515,6 +515,9 @@ public class DBXMLQueries {
* first query for Detections documents associated with this deployment and datablock.
* updated May 23
*/
if (dataBlock == null) {
return null;
}
String queryNoDepl = "{\"species\":{\"query\":{\"op\":\"lib:completename2tsn\",\"optype\":\"function\",\"operands\":[\"%s\"]},\"return\":{\"op\":\"lib:tsn2completename\",\"optype\":\"function\",\"operands\":[\"%s\"]}},\"return\":[\"Detections/Id\"],\"select\":[{\"op\":\"=\",\"operands\":[\"Detections/Algorithm/Software\",\"LongDataName\"],\"optype\":\"binary\"}],\"enclose\":1}";
String queryWithDepl = "{\"species\":{\"query\":{\"op\":\"lib:completename2tsn\",\"optype\":\"function\",\"operands\":[\"%s\"]},\"return\":{\"op\":\"lib:tsn2completename\",\"optype\":\"function\",\"operands\":[\"%s\"]}},\"return\":[\"Detections/Id\"],\"select\":[{\"op\":\"=\",\"operands\":[\"Detections/DataSource/DeploymentId\",\"TheDeploymentId\"],\"optype\":\"binary\"},{\"op\":\"=\",\"operands\":[\"Detections/Algorithm/Software\",\"LongDataName\"],\"optype\":\"binary\"}],\"enclose\":1}";
String query;

98
src/tethys/deployment/DeploymentHandler.java
View File

@ -73,6 +73,7 @@ import nilus.UnknownSensor;
import pamMaths.PamVector;
import pamMaths.STD;
import tethys.Collection;
import tethys.CollectionHandler;
import tethys.TethysControl;
import tethys.TethysLocationFuncs;
import tethys.TethysState;
@ -83,6 +84,7 @@ import tethys.TethysState.StateType;
import tethys.dbxml.DBXMLConnect;
import tethys.dbxml.TethysException;
import tethys.deployment.swing.DeploymentWizard;
import tethys.deployment.swing.EffortProblemDialog;
import tethys.deployment.swing.RecordingGapDialog;
import tethys.niluswraps.PDeployment;
import tethys.output.TethysExportParams;
@ -98,16 +100,9 @@ import tethys.swing.DeploymentTableObserver;
* @author dg50
*
*/
public class DeploymentHandler implements TethysStateObserver, DeploymentTableObserver {
public class DeploymentHandler extends CollectionHandler implements TethysStateObserver, DeploymentTableObserver {
private TethysControl tethysControl;
/**
* @return the tethysControl
*/
public TethysControl getTethysControl() {
return tethysControl;
}
// private TethysControl tethysControl;
private EffortFunctions effortFunctions;
@ -119,8 +114,11 @@ public class DeploymentHandler implements TethysStateObserver, DeploymentTableOb
private DeploymentExportOpts deploymentExportOptions = new DeploymentExportOpts();
public static final String helpPoint = "utilities.tethys.docs.deployments";
public DeploymentHandler(TethysControl tethysControl) {
super();
super(tethysControl, Collection.Deployments);
this.tethysControl = tethysControl;
@ -368,8 +366,39 @@ public class DeploymentHandler implements TethysStateObserver, DeploymentTableOb
public void createPamguardOverview() {
deploymentOverview = effortFunctions.makeRecordingOverview();
checkDeploymentOverview(deploymentOverview);
updateProjectDeployments();
matchPamguard2Tethys(deploymentOverview, projectDeployments);
tethysControl.sendStateUpdate(new TethysState(StateType.NEWPAMGUARDSELECTION));
}
/**
* Check the deployment overview for consistency.<br>
* Take the raw audio information and the binary information and check they are similar.
* if not, ask the user what to do.
* @param deploymentOverview
*/
private void checkDeploymentOverview(DeploymentOverview overview) {
RecordingList rawList = overview.getRawDataList();
RecordingList binList = overview.getBinaryDataList();
if (rawList == null || binList == null) {
return; // nothing to do
}
double similarity = rawList.getSimilarity(binList);
if (similarity > 0.95) {
return;
}
/*
* if we get here, it seems like the two lists are very different, so
* show a dialog to ask the user what to do.
*/
RecordingList selList = EffortProblemDialog.showDialog(tethysControl.getGuiFrame(), overview);
if (selList != null) {
tethysControl.getTethysExportParams().setEffortSourceName(selList.getSourceName());
}
}
/**
@ -381,22 +410,22 @@ public class DeploymentHandler implements TethysStateObserver, DeploymentTableOb
if (exportOptions != null) {
this.deploymentExportOptions = exportOptions;
deploymentOverview = getDeploymentOverview();
ArrayList<RecordingPeriod> allPeriods = deploymentOverview.getRecordingPeriods();
RecordingList allPeriods = deploymentOverview.getMasterList(getTethysControl());
exportDeployments(allPeriods);
}
}
/**
* Export deployments docs. Playing with a couple of different ways of doing this.
* @param selectedDeployments
* @param allPeriods
*/
public void exportDeployments(ArrayList<RecordingPeriod> selectedDeployments) {
public void exportDeployments(RecordingList allPeriods) {
TethysReporter.getTethysReporter().clear();
if (deploymentExportOptions.separateDeployments) {
exportSeparateDeployments(selectedDeployments);
exportSeparateDeployments(allPeriods);
}
else {
exportOneDeploymnet(selectedDeployments);
exportOneDeploymnet(allPeriods);
}
TethysReporter.getTethysReporter().showReport(tethysControl.getGuiFrame(), true);
}
@ -404,7 +433,7 @@ public class DeploymentHandler implements TethysStateObserver, DeploymentTableOb
/**
* Make one big deployment document with all the recording periods in it.
*/
private void exportOneDeploymnet(ArrayList<RecordingPeriod> selectedDeployments) {
private void exportOneDeploymnet(RecordingList recordingList) {
// do the lot, whatever ...
Float sampleRate = null;
AcquisitionControl daq = (AcquisitionControl) PamController.getInstance().findControlledUnit(AcquisitionControl.class, null);
@ -414,10 +443,9 @@ public class DeploymentHandler implements TethysStateObserver, DeploymentTableOb
sampleRate = daqParams.sampleRate;
}
selectedDeployments = getDeploymentOverview().getRecordingPeriods();
int freeId = getTethysControl().getDeploymentHandler().getFirstFreeDeploymentId();
RecordingPeriod onePeriod = new RecordingPeriod(selectedDeployments.get(0).getRecordStart(),
selectedDeployments.get(selectedDeployments.size()-1).getRecordStop());
RecordingPeriod onePeriod = new RecordingPeriod(recordingList.getStart(),
recordingList.getEnd());
TethysExportParams exportParams = tethysControl.getTethysExportParams();
String id = String.format("%s_%s", exportParams.getDatasetName(), "all");
Deployment deployment = createDeploymentDocument(freeId, onePeriod, id);
@ -425,7 +453,8 @@ public class DeploymentHandler implements TethysStateObserver, DeploymentTableOb
Deployment globalMeta = getTethysControl().getGlobalDeplopymentData();
deployment.setCruise(globalMeta.getCruise());
deployment.setSite(globalMeta.getSite());
if (selectedDeployments.size() > 1) {
ArrayList<RecordingPeriod> effortPeriods = recordingList.getEffortPeriods();
if (recordingList.size() > 1) {
// // now need to remove the sampling details - don't though, add invalid periods instead.
// SamplingDetails samplingDetails = deployment.getSamplingDetails();
// samplingDetails.getChannel().clear();
@ -440,9 +469,9 @@ public class DeploymentHandler implements TethysStateObserver, DeploymentTableOb
deployment.setQualityAssurance(qa = new AcousticDataQAType());
}
List<Quality> qualityList = qa.getQuality();
for (int i = 1; i < selectedDeployments.size(); i++) {
long end = selectedDeployments.get(i-1).getRecordStop();
long start = selectedDeployments.get(i).getRecordStart();
for (int i = 1; i < recordingList.size(); i++) {
long end = effortPeriods.get(i-1).getRecordStop();
long start = effortPeriods.get(i).getRecordStart();
Quality q = new Quality();
q.setStart(TethysTimeFuncs.xmlGregCalFromMillis(end));
q.setEnd(TethysTimeFuncs.xmlGregCalFromMillis(start));
@ -479,14 +508,15 @@ public class DeploymentHandler implements TethysStateObserver, DeploymentTableOb
/**
* Make a separate deployment document for every recording period.
*/
private void exportSeparateDeployments(ArrayList<RecordingPeriod> selectedDeployments) {
private void exportSeparateDeployments(RecordingList recordingList) {
int freeId = getTethysControl().getDeploymentHandler().getFirstFreeDeploymentId();
// fill in a few things from here
Deployment globalMeta = getTethysControl().getGlobalDeplopymentData();
TethysExportParams exportParams = tethysControl.getTethysExportParams();
for (int i = 0; i < selectedDeployments.size(); i++) {
RecordingPeriod recordPeriod = selectedDeployments.get(i);
ArrayList<RecordingPeriod> effortPeriods = recordingList.getEffortPeriods();
for (int i = 0; i < recordingList.size(); i++) {
RecordingPeriod recordPeriod = effortPeriods.get(i);
PDeployment exDeploymnet = recordPeriod.getMatchedTethysDeployment();
Deployment deployment = null;
String id = String.format("%s_%d", exportParams.getDatasetName(), i);
@ -532,8 +562,9 @@ public class DeploymentHandler implements TethysStateObserver, DeploymentTableOb
if (deployments == null || deploymentOverview == null) {
return;
}
ArrayList<RecordingPeriod> recordingPeriods = deploymentOverview.getRecordingPeriods();
for (RecordingPeriod aPeriod : recordingPeriods) {
RecordingList recordingList = deploymentOverview.getMasterList(getTethysControl());
ArrayList<RecordingPeriod> effortPeriods = recordingList.getEffortPeriods();
for (RecordingPeriod aPeriod : effortPeriods) {
PDeployment closestDeployment = findClosestDeployment(aPeriod, deployments);
aPeriod.setMatchedTethysDeployment(closestDeployment);
if (closestDeployment != null) {
@ -592,7 +623,8 @@ public class DeploymentHandler implements TethysStateObserver, DeploymentTableOb
if (deploymentOverview == null) {
return matched;
}
for (RecordingPeriod period : deploymentOverview.getRecordingPeriods()) {
ArrayList<RecordingPeriod> effortPeriods = deploymentOverview.getMasterList(getTethysControl()).getEffortPeriods();
for (RecordingPeriod period : effortPeriods) {
PDeployment deployment = period.getMatchedTethysDeployment();
if (deployment != null) {
if (matched.contains(deployment) == false) {
@ -1230,7 +1262,8 @@ public class DeploymentHandler implements TethysStateObserver, DeploymentTableOb
}
regimens.add(regimen);
DutyCycleInfo dutyCycleInf = deploymentOverview.getDutyCycleInfo();
RecordingList recordingList = deploymentOverview.getMasterList(getTethysControl());
DutyCycleInfo dutyCycleInf = recordingList.assessDutyCycle();
boolean isDS = dutyCycleInf != null && dutyCycleInf.isDutyCycled;
if (isDS) {
DutyCycle dutyCycle = new DutyCycle();
@ -1307,4 +1340,9 @@ public class DeploymentHandler implements TethysStateObserver, DeploymentTableOb
return deploymentExportOptions;
}
@Override
public String getHelpPoint() {
return helpPoint;
}
}

110
src/tethys/deployment/DeploymentOverview.java
View File

@ -1,16 +1,6 @@
package tethys.deployment;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.ListIterator;
import Acquisition.AcquisitionControl;
import Acquisition.AcquisitionParameters;
import Acquisition.DaqStatusDataUnit;
import PamController.PamControlledUnit;
import PamController.PamController;
import PamguardMVC.PamDataBlock;
import tethys.TethysControl;
/**
* Class to give a general overview of all the effort in PAMGuard which will form the
@ -22,59 +12,83 @@ import PamguardMVC.PamDataBlock;
*/
public class DeploymentOverview {
private ArrayList<RecordingPeriod> recordingPeriods = new ArrayList<>();
private RecordingList rawDataList;
private DutyCycleInfo dutyCycleInfo;
private RecordingList binaryDataList;
// private DutyCycleInfo dutyCycleInfo;
public DeploymentOverview(DutyCycleInfo dutyCycleInfo) {
super();
this.dutyCycleInfo = dutyCycleInfo;
}
// public DeploymentOverview(DutyCycleInfo dutyCycleInfo) {
// super();
// this.dutyCycleInfo = dutyCycleInfo;
// }
public DeploymentOverview(DutyCycleInfo dutyCycleInfo, ArrayList<RecordingPeriod> tempPeriods) {
this.dutyCycleInfo = dutyCycleInfo;
this.recordingPeriods = tempPeriods;
}
public void addRecordingPeriod(long start, long stop) {
addRecordingPeriod(new RecordingPeriod(start, stop));
public DeploymentOverview(DutyCycleInfo dutyCycleInfo, RecordingList rawDataList, RecordingList binaryDataList) {
// this.dutyCycleInfo = dutyCycleInfo;
this.rawDataList = rawDataList;
this.binaryDataList = binaryDataList;
}
private void addRecordingPeriod(RecordingPeriod recordingPeriod) {
recordingPeriods.add(recordingPeriod);
/**
* @return the rawDataList
*/
public RecordingList getRawDataList() {
return rawDataList;
}
public ArrayList<RecordingPeriod> getRecordingPeriods() {
return recordingPeriods;
/**
* @return the binaryDataList
*/
public RecordingList getBinaryDataList() {
return binaryDataList;
}
public DutyCycleInfo getDutyCycleInfo() {
return dutyCycleInfo;
// /**
// * @return the dutyCycleInfo
// */
// public DutyCycleInfo getDutyCycleInfo() {
// return dutyCycleInfo;
// }
public RecordingList getMasterList(TethysControl tethysControl) {
return getMasterList(tethysControl.getTethysExportParams().getEffortSourceName());
}
public RecordingList getMasterList(String effortSourceName) {
if (effortSourceName == null) {
return getLongestList();
}
if (binaryDataList != null & binaryDataList.getSourceName().equals(effortSourceName)) {
return binaryDataList;
}
if (rawDataList != null & rawDataList.getSourceName().equals(effortSourceName)) {
return rawDataList;
}
return getLongestList();
}
/**
* Get the start time of the first recording
* Get the recording list with the greatest duration (start to end)
* not looking at coverage between those times.
* @return
*/
public Long getFirstStart() {
if (recordingPeriods.size() > 0) {
return recordingPeriods.get(0).getRecordStart();
public RecordingList getLongestList() {
if (binaryDataList == null) {
return rawDataList;
}
return null;
}
/**
* Get the end time of the last recording
* @return
*/
public Long getLastEnd() {
if (recordingPeriods.size() > 0) {
return recordingPeriods.get(recordingPeriods.size()-1).getRecordStop();
if (rawDataList == null) {
return binaryDataList;
}
long lRaw = rawDataList.duration();
long lBin = binaryDataList.duration();
if (lRaw > lBin) {
return rawDataList;
}
else {
return binaryDataList;
}
return null;
}
}

242
src/tethys/deployment/EffortFunctions.java
View File

@ -41,36 +41,40 @@ public class EffortFunctions {
this.tethysControl = tethysControl;
}
private DeploymentOverview createOverview(RecordingList tempPeriods) {
DutyCycleInfo dutyCycleinfo = assessDutyCycle(tempPeriods);
if (dutyCycleinfo == null) {
return null;
}
// if it's duty cycles, then we only want a single entry.
RecordingList deploymentPeriods;
if (dutyCycleinfo.isDutyCycled == false) {
deploymentPeriods = tempPeriods;
}
else {
deploymentPeriods = new RecordingList();
deploymentPeriods.add(new RecordingPeriod(tempPeriods.get(0).getRecordStart(), tempPeriods.get(tempPeriods.size()-1).getRecordStop()));
}
/*
* do another sort of the deploymentPeriods. The start stops were in the order they went into the
* database in the hope that pairs were the right way round. Now check all data are/
*/
Collections.sort(deploymentPeriods, new Comparator<RecordingPeriod>() {
@Override
public int compare(RecordingPeriod o1, RecordingPeriod o2) {
return (int) (o1.getRecordStart()-o2.getRecordStart());
}
});
DeploymentOverview deploymentOverview = new DeploymentOverview(dutyCycleinfo, deploymentPeriods);
return deploymentOverview;
}
// private DeploymentOverview createOverview(RecordingList tempPeriods) {
//
// tempPeriods.sort();
//
// DutyCycleInfo dutyCycleinfo = assessDutyCycle(tempPeriods);
// if (dutyCycleinfo == null) {
// return null;
// }
//
//
// // if it's duty cycles, then we only want a single entry.
// RecordingList deploymentPeriods;
// if (dutyCycleinfo.isDutyCycled == false) {
// deploymentPeriods = tempPeriods;
// }
// else {
// deploymentPeriods = new RecordingList(tempPeriods.getSourceName());
// deploymentPeriods.add(new RecordingPeriod(tempPeriods.get(0).getRecordStart(), tempPeriods.get(tempPeriods.size()-1).getRecordStop()));
// }
// /*
// * do another sort of the deploymentPeriods. The start stops were in the order they went into the
// * database in the hope that pairs were the right way round. Now check all data are/
// */
// deploymentPeriods.sort();
//// Collections.sort(deploymentPeriods, new Comparator<RecordingPeriod>() {
//// @Override
//// public int compare(RecordingPeriod o1, RecordingPeriod o2) {
//// return (int) (o1.getRecordStart()-o2.getRecordStart());
//// }
//// });
//
// DeploymentOverview deploymentOverview = new DeploymentOverview(dutyCycleinfo, deploymentPeriods);
// return deploymentOverview;
// }
public DeploymentOverview makeRecordingOverview() {
@ -79,13 +83,18 @@ public class EffortFunctions {
RecordingList binaryPeriods = listBinaryFiles();
long l1 = listDuration(recordingPeriods);
long l2 = listDuration(binaryPeriods);
if (listDuration(binaryPeriods) > listDuration(recordingPeriods)) {
recordingPeriods = binaryPeriods;
}
// see what the similarity is between them
// double sim = recordingPeriods.getSimilarity(binaryPeriods);
// double testSim = recordingPeriods.getSimilarity(recordingPeriods);
DeploymentOverview deploymentOverview = createOverview(recordingPeriods);
// long l1 = listDuration(recordingPeriods);
// long l2 = listDuration(binaryPeriods);
// if (listDuration(binaryPeriods) > listDuration(recordingPeriods)) {
// recordingPeriods = binaryPeriods;
// }
//
// DeploymentOverview deploymentOverview = createOverview(recordingPeriods);
DeploymentOverview deploymentOverview = new DeploymentOverview(null, recordingPeriods, binaryPeriods);
return deploymentOverview;
}
@ -128,7 +137,8 @@ public class EffortFunctions {
}
}
}
bestList = mergeRecordings(bestList);
DeploymentExportOpts exportOptions = tethysControl.getDeploymentHandler().getDeploymentExportOptions();
bestList.mergeRecordingPeriods(exportOptions.maxRecordingGapSeconds*1000);
return bestList;
}
@ -138,7 +148,7 @@ public class EffortFunctions {
if (mapPoints == null) {
return null;
}
RecordingList periods = new RecordingList();
RecordingList periods = new RecordingList(dataMap.getDataMapName());
for (OfflineDataMapPoint mapPoint : mapPoints) {
periods.add(new RecordingPeriod(mapPoint.getStartTime(), mapPoint.getEndTime()));
}
@ -219,117 +229,60 @@ public class EffortFunctions {
// PamCalendar.formatDBDateTime(aP.getRecordStop()));
// }
tempPeriods = mergeRecordings(tempPeriods);
return tempPeriods;
}
/**
* Merge close recordings and discard ones that are too short.
* @param tempPeriods all recording periods, may be from consecutive files.
* @return merged list.
*/
private RecordingList mergeRecordings(RecordingList tempPeriods) {
// now go through those and merge into longer periods where there is no gap between files.
if (tempPeriods == null) {
return null;
}
tempPeriods.sort();
DeploymentExportOpts exportOptions = tethysControl.getDeploymentHandler().getDeploymentExportOptions();
ListIterator<RecordingPeriod> iterator = tempPeriods.listIterator();
RecordingPeriod prevPeriod = null;
while (iterator.hasNext()) {
RecordingPeriod nextPeriod = iterator.next();
long nextDur = nextPeriod.getRecordStop()-nextPeriod.getRecordStart();
if (nextDur == 0) {
continue;
}
if (prevPeriod != null) {
long gap = nextPeriod.getRecordStart() - prevPeriod.getRecordStop();
long prevDur = prevPeriod.getRecordStop()-prevPeriod.getRecordStart();
if (gap < exportOptions.maxRecordingGapSeconds*1000) {
// ignoring up to 3s gap or a sample error < 2%.Dunno if this is sensible or not.
prevPeriod.setRecordStop(nextPeriod.getRecordStop());
iterator.remove();
nextPeriod = prevPeriod;
}
}
prevPeriod = nextPeriod;
}
// now remove ones which are too short even after merging.
iterator = tempPeriods.listIterator();
while (iterator.hasNext()) {
RecordingPeriod nextPeriod = iterator.next();
long duration = nextPeriod.getDuration();
if (duration < exportOptions.minRecordingLengthSeconds*1000L) {
iterator.remove();
}
}
tempPeriods.mergeRecordingPeriods(exportOptions.maxRecordingGapSeconds*1000);
return tempPeriods;
}
/**
* Work out whether or not the data are evenly duty cycled by testing the
* distributions of on and off times.
* @param tempPeriods
* @return
*/
private DutyCycleInfo assessDutyCycle(RecordingList tempPeriods) {
if (tempPeriods == null) {
return null;
}
int n = tempPeriods.size();
if (n < 2) {
return new DutyCycleInfo(false, 0,0,n);
}
double[] ons = new double[n-1]; // ignore the last one since it may be artificially shortened which is OK
double[] gaps = new double[n-1];
for (int i = 0; i < n-1; i++) {
ons[i] = tempPeriods.get(i).getDuration()/1000.;
gaps[i] = (tempPeriods.get(i+1).getRecordStart()-tempPeriods.get(i).getRecordStop())/1000.;
}
/* now look at how consistent those values are
* But some data gets messed by small gaps, so want to
* remove outliers and concentrate on say 80% of the data.
*/
ons = getDistributionCentre(ons, 80);
gaps = getDistributionCentre(gaps, 80);
Arrays.sort(gaps);
// /**
// * Merge close recordings and discard ones that are too short.
// * @param tempPeriods all recording periods, may be from consecutive files.
// * @return merged list.
// */
// private RecordingList mergeRecordings(RecordingList tempPeriods) {
// // now go through those and merge into longer periods where there is no gap between files.
// if (tempPeriods == null) {
// return null;
// }
//
// DeploymentExportOpts exportOptions = tethysControl.getDeploymentHandler().getDeploymentExportOptions();
//
// ListIterator<RecordingPeriod> iterator = tempPeriods.listIterator();
// RecordingPeriod prevPeriod = null;
// while (iterator.hasNext()) {
// RecordingPeriod nextPeriod = iterator.next();
// long nextDur = nextPeriod.getRecordStop()-nextPeriod.getRecordStart();
// if (nextDur == 0) {
// continue;
// }
// if (prevPeriod != null) {
// long gap = nextPeriod.getRecordStart() - prevPeriod.getRecordStop();
// long prevDur = prevPeriod.getRecordStop()-prevPeriod.getRecordStart();
// if (gap < exportOptions.maxRecordingGapSeconds*1000) {
// // ignoring up to 3s gap or a sample error < 2%.Dunno if this is sensible or not.
// prevPeriod.setRecordStop(nextPeriod.getRecordStop());
// iterator.remove();
// nextPeriod = prevPeriod;
// }
// }
// prevPeriod = nextPeriod;
// }
// // now remove ones which are too short even after merging.
// iterator = tempPeriods.listIterator();
// while (iterator.hasNext()) {
// RecordingPeriod nextPeriod = iterator.next();
// long duration = nextPeriod.getDuration();
// if (duration < exportOptions.minRecordingLengthSeconds*1000L) {
// iterator.remove();
// }
// }
//
// return tempPeriods;
// }
STD std = new STD();
double onsMean = std.getMean(ons);
double onsSTD = std.getSTD(ons);
double gapsMean = std.getMean(gaps);
double gapsSTD = std.getSTD(gaps);
boolean dutyCycle = onsSTD/onsMean < .05 && gapsSTD/gapsMean < 0.05;
DutyCycleInfo cycleInfo = new DutyCycleInfo(dutyCycle, onsMean, gapsMean, tempPeriods.size());
return cycleInfo;
}
/**
* Get the central part of a distribution without any outliers so
* that we can get a better assessment of duty cycle.
* @param data unsorted distribution data.
* @param percent percentage to include (half this removed from top and bottom)
* @return
*/
private double[] getDistributionCentre(double[] data, double percent) {
if (data == null) {
return null;
}
Arrays.sort(data);
int nRem = (int) Math.round(data.length * (100-percent)/200);
int newLen = data.length-nRem*2;
double[] subdata = Arrays.copyOfRange(data, nRem, data.length-2*nRem);
if (subdata.length < 2) {
return data;
}
return subdata;
}
/**
* Get data times from any other datamap, since this will generally match the acquisition anyway
@ -360,16 +313,17 @@ public class EffortFunctions {
return null;
}
// get the times out of it.
RecordingList recPeriods = new RecordingList();
RecordingList recPeriods = new RecordingList(bestMap.getDataMapName());
List<OfflineDataMapPoint> mapPoints = bestMap.getMapPoints();
for (OfflineDataMapPoint mapPoint : mapPoints) {
recPeriods.add(new RecordingPeriod(mapPoint.getStartTime(), mapPoint.getEndTime()));
recPeriods.add(mapPoint.getStartTime(), mapPoint.getEndTime());
}
return recPeriods;
}
private RecordingList extractTimesFromStatus(ArrayList<DaqStatusDataUnit> allStatusData) {
RecordingList tempPeriods = new RecordingList();
RecordingList tempPeriods = new RecordingList("Data acquisition status");
long dataStart = Long.MAX_VALUE;
long dataEnd = Long.MIN_VALUE;
Long lastStart = null;

211
src/tethys/deployment/RecordingList.java
View File

@ -1,12 +1,43 @@
package tethys.deployment;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.Comparator;
import java.util.Iterator;
public class RecordingList extends ArrayList<RecordingPeriod> {
import PamUtils.PamCalendar;
import pamMaths.STD;
/**
* Information about periods of effort that might come from either the raw data recordings or
* an analysis of binary data maps.
* @author dg50
*
*/
public class RecordingList implements Serializable {
private static final long serialVersionUID = 1L;
private ArrayList<RecordingPeriod> effortPeriods = new ArrayList();
/**
* Name / source of this list.
*/
private String sourceName;
/**
* @param sourceName
*/
public RecordingList(String sourceName) {
this.sourceName = sourceName;
}
public RecordingList(String sourceName, ArrayList<RecordingPeriod> selectedDeployments) {
this.sourceName = sourceName;
this.effortPeriods = selectedDeployments;
}
/**
* Get the duration of the recording periods from start to end.
@ -21,27 +52,27 @@ public class RecordingList extends ArrayList<RecordingPeriod> {
* @return
*/
public long getStart() {
if (size() == 0) {
if (effortPeriods.size() == 0) {
return 0;
}
return get(0).getRecordStart();
return effortPeriods.get(0).getRecordStart();
}
/**
* get the end of the last in the list.
*/
public long getEnd() {
if (size() == 0) {
if (effortPeriods.size() == 0) {
return 0;
}
return get(size()-1).getRecordStop();
return effortPeriods.get(effortPeriods.size()-1).getRecordStop();
}
/**
* Sort the list in ascending order.
*/
public void sort() {
Collections.sort(this, new Comparator<RecordingPeriod>() {
Collections.sort(effortPeriods, new Comparator<RecordingPeriod>() {
@Override
public int compare(RecordingPeriod o1, RecordingPeriod o2) {
@ -49,4 +80,172 @@ public class RecordingList extends ArrayList<RecordingPeriod> {
}
});
}
/**
* Get the coverage as a fraction. This is the sum of the individual periods divided
* by the start to end times
* @return
*/
public double getCoverage() {
long cov = 0;
long durTot = 0;
if (effortPeriods.size() == 0) {
return 0;
}
Iterator<RecordingPeriod> it = effortPeriods.iterator();
while (it.hasNext()) {
RecordingPeriod rp = it.next();
cov += rp.getDuration();
}
durTot = getEnd()-getStart();
return (double) cov / (double) durTot;
}
/**
* Merge recording periods, with a max gap between periods in milliseconds.
* @param maxGap
* @return the number of periods removed.
*/
public int mergeRecordingPeriods(long maxGap) {
if (effortPeriods.size() < 2) {
return 0;
}
Iterator<RecordingPeriod> it = effortPeriods.iterator();
RecordingPeriod prev = it.next();
int removed = 0;
while (it.hasNext()) {
RecordingPeriod curr = it.next();
if (curr.getRecordStart() - prev.getRecordStop() <= maxGap) {
prev.setRecordStop(curr.getRecordStop());
it.remove();
removed++;
}
else {
prev = curr;
}
}
return removed;
}
/**
* Work out whether or not the data are evenly duty cycled by testing the
* distributions of on and off times.
* @param tempPeriods
* @return
*/
public DutyCycleInfo assessDutyCycle() {
if (effortPeriods == null) {
return null;
}
int n = effortPeriods.size();
if (n < 2) {
return new DutyCycleInfo(false, 0,0,n);
}
double[] ons = new double[n-1]; // ignore the last one since it may be artificially shortened which is OK
double[] gaps = new double[n-1];
for (int i = 0; i < n-1; i++) {
ons[i] = effortPeriods.get(i).getDuration()/1000.;
gaps[i] = (effortPeriods.get(i+1).getRecordStart()-effortPeriods.get(i).getRecordStop())/1000.;
}
/* now look at how consistent those values are
* But some data gets messed by small gaps, so want to
* remove outliers and concentrate on say 80% of the data.
*/
ons = getDistributionCentre(ons, 80);
gaps = getDistributionCentre(gaps, 80);
Arrays.sort(gaps);
STD std = new STD();
double onsMean = std.getMean(ons);
double onsSTD = std.getSTD(ons);
double gapsMean = std.getMean(gaps);
double gapsSTD = std.getSTD(gaps);
boolean dutyCycle = onsSTD/onsMean < .05 && gapsSTD/gapsMean < 0.05;
DutyCycleInfo cycleInfo = new DutyCycleInfo(dutyCycle, onsMean, gapsMean, effortPeriods.size());
return cycleInfo;
}
/**
* Get the central part of a distribution without any outliers so
* that we can get a better assessment of duty cycle.
* @param data unsorted distribution data.
* @param percent percentage to include (half this removed from top and bottom)
* @return
*/
private double[] getDistributionCentre(double[] data, double percent) {
if (data == null) {
return null;
}
Arrays.sort(data);
int nRem = (int) Math.round(data.length * (100-percent)/200);
int newLen = data.length-nRem*2;
double[] subdata = Arrays.copyOfRange(data, nRem, data.length-2*nRem);
if (subdata.length < 2) {
return data;
}
return subdata;
}
/**
* @return the sourceName
*/
public String getSourceName() {
return sourceName;
}
@Override
public String toString() {
if (effortPeriods.size() == 0) {
return "Empty recording list";
}
String str = String.format("%s: %s to %s, %3.1f%% coverage", getSourceName(),
PamCalendar.formatDBDateTime(getStart()),
PamCalendar.formatDBDateTime(getEnd()), getCoverage()*100);
return str;
}
/**
* Get similarity to another recording list. 1 = identical, 0 means not even overlapping.
* @param other other recording list.
* @return measure of similarity.
*/
public double getSimilarity(RecordingList other) {
double sim1 = (double) other.duration() / (double) this.duration();
if (sim1 > 1) {
sim1 = 1./sim1;
}
long overlap = Math.min(other.getEnd(), this.getEnd()) - Math.max(other.getStart(), this.getStart());
overlap = Math.max(0, overlap);
long longest = Math.max(other.duration(), this.duration());
double sim2 = (double) overlap / (double) longest;
return Math.min(sim1, sim2);
}
/**
* Add a recording period to the list.
* @param recordingPeriod
*/
public void add(RecordingPeriod recordingPeriod) {
effortPeriods.add(recordingPeriod);
}
/**
* Add a recording period to the list.
* @param startTime
* @param endTime
*/
public void add(long startTime, long endTime) {
add (new RecordingPeriod(startTime, endTime));
}
public int size() {
return effortPeriods.size();
}
/**
* @return the effortPeriods
*/
public ArrayList<RecordingPeriod> getEffortPeriods() {
return effortPeriods;
}
}

7
src/tethys/deployment/RecordingPeriod.java
View File

@ -1,5 +1,6 @@
package tethys.deployment;
import PamUtils.PamCalendar;
import tethys.niluswraps.PDeployment;
public class RecordingPeriod {
@ -71,5 +72,11 @@ public class RecordingPeriod {
selected = !selected;
return selected;
}
@Override
public String toString() {
return String.format("%s to %s, %s", PamCalendar.formatDBDateTime(recordStart),
PamCalendar.formatDBDateTime(recordStop), PamCalendar.formatDuration(getDuration()));
}
}

163
src/tethys/deployment/swing/EffortProblemDialog.java Normal file
View File

@ -0,0 +1,163 @@
package tethys.deployment.swing;
import java.awt.BorderLayout;
import java.awt.GridBagConstraints;
import java.awt.GridBagLayout;
import java.awt.GridLayout;
import java.awt.Window;
import javax.swing.BoxLayout;
import javax.swing.ButtonGroup;
import javax.swing.JCheckBox;
import javax.swing.JLabel;
import javax.swing.JPanel;
import javax.swing.JRadioButton;
import javax.swing.border.TitledBorder;
import PamUtils.PamCalendar;
import PamView.dialog.PamDialog;
import PamView.dialog.PamGridBagContraints;
import tethys.deployment.DeploymentOverview;
import tethys.deployment.RecordingList;
/**
* Handle problems when binary and raw effort don't add up
* @author dg50
*
*/
public class EffortProblemDialog extends PamDialog {
private JRadioButton useRaw, useBinary, useNeither;
private JLabel generalInfo;
private InfoSet[] infoSets = new InfoSet[2];
private RecordingList chosenList;
private DeploymentOverview deploymentOverview;
private static EffortProblemDialog singleInstance;
private static final String[] setNames = {"Raw data", "Binary data"};
private EffortProblemDialog(Window parentFrame) {
super(parentFrame, "Deployment Effort", false);
JPanel mainPanel = new JPanel();
mainPanel.setLayout(new BorderLayout());
mainPanel.setBorder(new TitledBorder("Effort information"));
String info = "<html>There is a mismatch between the time period covered by the raw<br>"
+ "data recordings and the time covered in the binary data.<br> "
+ "Select the one you wish to use, or Cancel and sort out your data<br>"
+ "prior to restarting the Tethys export process</html>";
generalInfo = new JLabel(info);
// generalInfo.setBorder(new TitledBorder("General"));
mainPanel.add(generalInfo, BorderLayout.NORTH);
JPanel botPanel = new JPanel(new GridLayout(2, 1));
mainPanel.add(botPanel, BorderLayout.CENTER);
ButtonGroup bg = new ButtonGroup();
for (int i = 0; i < 2; i++) {
GridBagConstraints c = new PamGridBagContraints();
JPanel subPanel = new JPanel(new GridBagLayout());
botPanel.add(subPanel);
infoSets[i] = new InfoSet(setNames[i]);
c.gridwidth = 2;
subPanel.add(infoSets[i].name, c);
c.gridx += c.gridwidth;
subPanel.add(infoSets[i].select, c);
c.gridx = 0;
c.gridy++;
c.gridwidth = 1;
subPanel.add(new JLabel("Start: ", JLabel.RIGHT), c);
c.gridx++;
subPanel.add(infoSets[i].start, c);
c.gridx++;
subPanel.add(new JLabel("End: ", JLabel.RIGHT), c);
c.gridx++;
subPanel.add(infoSets[i].end, c);
c.gridy++;
c.gridx = 0;
subPanel.add(new JLabel("Duration: ", JLabel.RIGHT), c);
c.gridx++;
subPanel.add(infoSets[i].duration, c);
c.gridx++;
subPanel.add(new JLabel("Coverage: ", JLabel.RIGHT), c);
c.gridx++;
subPanel.add(infoSets[i].occupancy, c);
bg.add(infoSets[i].select);
}
setDialogComponent(mainPanel);
setResizable(true);
}
public static RecordingList showDialog(Window parentFrame, DeploymentOverview deploymentOverview) {
singleInstance = new EffortProblemDialog(parentFrame);
singleInstance.setData(deploymentOverview);
singleInstance.setVisible(true);
return singleInstance.chosenList;
}
private void setData(DeploymentOverview deploymentOverview) {
this.deploymentOverview = deploymentOverview;
RecordingList rl;
for (int i = 0; i < 2; i++) {
if (i == 0) {
rl = deploymentOverview.getRawDataList();
}
else {
rl = deploymentOverview.getBinaryDataList();
}
infoSets[i].start.setText(PamCalendar.formatDBDateTime(rl.getStart()));
infoSets[i].end.setText(PamCalendar.formatDBDateTime(rl.getEnd()));
infoSets[i].duration.setText(PamCalendar.formatDuration(rl.duration()));
infoSets[i].occupancy.setText(String.format("%3.0f%%", rl.getCoverage()*100.));
}
invalidate();
pack();
}
@Override
public boolean getParams() {
if (infoSets[0].select.isSelected()) {
chosenList = deploymentOverview.getRawDataList();
return true;
}
if (infoSets[1].select.isSelected()) {
chosenList = deploymentOverview.getBinaryDataList();
return true;
}
return false;
}
@Override
public void cancelButtonPressed() {
// TODO Auto-generated method stub
}
@Override
public void restoreDefaultSettings() {
// TODO Auto-generated method stub
}
private class InfoSet {
JLabel name, start, end, duration, occupancy;
JCheckBox select;
/**
*
*/
public InfoSet(String name) {
super();
this.name = new JLabel(name);
this.start = new JLabel(" ");
this.end = new JLabel(" ");
this.select = new JCheckBox("Select " + name);
duration = new JLabel(" ");
occupancy = new JLabel(" ");
}
}
}

7
src/tethys/deployment/swing/ProjectInformationPanel.java
View File

@ -23,6 +23,7 @@ import nilus.Deployment;
import tethys.TethysControl;
import tethys.swing.NewProjectDialog;
import tethys.swing.SelectProjectDialog;
import tethys.tooltips.TethysTips;
/**
* Panel for entering project information
@ -109,6 +110,12 @@ public class ProjectInformationPanel {
});
}
project.setToolTipText(TethysTips.findTip(Deployment.class, "Project"));
cruise.setToolTipText(TethysTips.findTip(Deployment.class, "Cruise"));
region.setToolTipText(TethysTips.findTip(Deployment.class, "Region"));
site.setToolTipText(TethysTips.findTip(Deployment.class, "Site"));
}
/**

3
src/tethys/deployment/swing/RecordingGapDialog.java
View File

@ -36,7 +36,7 @@ public class RecordingGapDialog extends PamDialog {
c.gridx++;
mainPanel.add(new JLabel(" seconds", JLabel.RIGHT), c);
maxGap.setToolTipText("Maximum gap between recording periods. Periods with a gap less than this will be counted as one");
maxGap.setToolTipText("Maximum gap between recording periods. Sequential periods with a gap less than this will be counted as one");
minLength.setToolTipText("Minimum recording length. Recording sections shorter than this will be ignored");
setDialogComponent(mainPanel);
@ -80,6 +80,7 @@ public class RecordingGapDialog extends PamDialog {
@Override
public void restoreDefaultSettings() {
DeploymentExportOpts defaults = new DeploymentExportOpts();
setParams(defaults);
}
}

12
src/tethys/detection/BinnedGranularityHandler.java
View File

@ -6,11 +6,16 @@ import java.util.Arrays;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map.Entry;
import javax.xml.datatype.DatatypeConstants;
import javax.xml.datatype.XMLGregorianCalendar;
import java.util.Set;
import PamguardMVC.PamDataBlock;
import PamguardMVC.PamDataUnit;
import nilus.Detection;
import nilus.Detections;
import nilus.SpeciesIDType;
import tethys.TethysControl;
import tethys.TethysTimeFuncs;
@ -54,6 +59,8 @@ public class BinnedGranularityHandler extends GranularityHandler {
public void prepare(long timeMillis) {
// long binStart = DetectionsHandler.roundDownBinStart(timeMillis, binDurationMillis);
// startBin(binStart);
// startBin(timeMillis);
currentDetections.clear();
}
// private void startBin(long timeMillis) {
@ -169,4 +176,9 @@ public class BinnedGranularityHandler extends GranularityHandler {
return closeBins(timeMillis);
}
@Override
protected boolean autoEffortFix(Detections detections, Detection det) {
return contractDetection(detections, det);
}
}

6
src/tethys/detection/CallGranularityHandler.java
View File

@ -3,6 +3,7 @@ package tethys.detection;
import PamguardMVC.PamDataBlock;
import PamguardMVC.PamDataUnit;
import nilus.Detection;
import nilus.Detections;
import tethys.TethysControl;
import tethys.output.StreamExportParams;
import tethys.output.TethysExportParams;
@ -37,4 +38,9 @@ public class CallGranularityHandler extends GranularityHandler {
return null;
}
@Override
protected boolean autoEffortFix(Detections detections, Detection det) {
return expandEffort(detections, det);
}
}

17
src/tethys/detection/DetectionExportProgress.java
View File

@ -6,10 +6,11 @@ import tethys.niluswraps.PDeployment;
public class DetectionExportProgress {
public static final int STATE_GATHERING = 1;
public static final int STATE_CANCELED = 2;
public static final int STATE_COMPLETE = 3;
public static final int STATE_WRITING = 4;
public static final int STATE_COUNTING = 5;
public static final int STATE_COUNTING = 2;
public static final int STATE_WRITING = 3;
public static final int STATE_CANCELED = 4;
public static final int STATE_COMPLETE = 5;
public PDeployment currentDeployment;
public Detections currentDetections;
public long lastUnitTime;
@ -17,12 +18,18 @@ public class DetectionExportProgress {
public int exportCount;
public int skipCount;
public int state;
public int totalDeployments, deploymentsDone;
public int nMapPoints;
public int doneMapPoints;
public DetectionExportProgress(PDeployment currentDeployment, Detections currentDetections, long lastUnitTime,
public DetectionExportProgress(PDeployment currentDeployment, Detections currentDetections, int nMapPoints, int doneMapPoints,
long lastUnitTime,
long totalUnits, int exportCount, int skipCount, int state) {
super();
this.currentDeployment = currentDeployment;
this.currentDetections = currentDetections;
this.nMapPoints = nMapPoints;
this.doneMapPoints = doneMapPoints;
this.lastUnitTime = lastUnitTime;
this.totalUnits = totalUnits;
this.exportCount = exportCount;

157
src/tethys/detection/DetectionsHandler.java
View File

@ -1,9 +1,13 @@
package tethys.detection;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.ListIterator;
import javax.swing.SwingWorker;
import javax.xml.datatype.DatatypeConstants;
import javax.xml.datatype.XMLGregorianCalendar;
import PamController.PamControlledUnit;
import PamController.PamController;
@ -32,6 +36,7 @@ import nilus.Detections;
import nilus.GranularityEnumType;
import nilus.Helper;
import tethys.Collection;
import tethys.CollectionHandler;
import tethys.TethysControl;
import tethys.TethysTimeFuncs;
import tethys.dbxml.DBXMLConnect;
@ -39,7 +44,6 @@ import tethys.dbxml.TethysException;
import tethys.deployment.DeploymentHandler;
import tethys.niluswraps.PDeployment;
import tethys.niluswraps.PDetections;
import tethys.output.DatablockSynchInfo;
import tethys.output.StreamExportParams;
import tethys.output.TethysExportParams;
import tethys.pamdata.TethysDataProvider;
@ -54,9 +58,7 @@ import tethys.swing.export.DetectionsExportWizard;
* @author dg50
*
*/
public class DetectionsHandler {
private TethysControl tethysControl;
public class DetectionsHandler extends CollectionHandler {
public int uniqueDetectionsId=1;
public int uniqueDetectionId;
@ -65,12 +67,14 @@ public class DetectionsHandler {
private ExportWorker exportWorker;
public static final String helpPoint = "utilities.tethys.docs.detect_localize";
/**
*
* @param tethysControl
*/
public DetectionsHandler(TethysControl tethysControl) {
super();
super(tethysControl, Collection.Detections);
this.tethysControl = tethysControl;
}
@ -296,9 +300,14 @@ public class DetectionsHandler {
viewerLoadPolicy = ViewerLoadPolicy.LOAD_UTCNORMAL;
}
GranularityHandler granularityHandler = GranularityHandler.getHandler(streamExportParams.granularity, tethysControl, dataBlock, exportParams, streamExportParams);
int totalMaps = 0;
int totalMappedPoints = 0;
int totalLoadedDatas = 0;
int totalMapPoints = dataMap.getNumMapPoints();
int doneMapPoints = 0;
for (PDeployment deployment : deployments) {
int documentCount = 0;
prog = new DetectionExportProgress(deployment, null,
prog = new DetectionExportProgress(deployment, null, totalMapPoints, doneMapPoints,
lastUnitTime, totalCount, exportCount, skipCount, DetectionExportProgress.STATE_COUNTING);
exportObserver.update(prog);
granularityHandler.prepare(deployment.getAudioStart());
@ -308,9 +317,10 @@ public class DetectionsHandler {
for (OfflineDataMapPoint mapPoint : mapPoints) {
if (!activeExport) {
prog = new DetectionExportProgress(deployment, null,
prog = new DetectionExportProgress(deployment, null,totalMapPoints, doneMapPoints,
lastUnitTime, totalCount, exportCount, skipCount, DetectionExportProgress.STATE_CANCELED);
exportObserver.update(prog);
break;
}
if (mapPoint.getEndTime() < deployment.getAudioStart()) {
@ -319,10 +329,13 @@ public class DetectionsHandler {
if (mapPoint.getStartTime() >= deployment.getAudioEnd()) {
break;
}
totalMaps ++;
totalMappedPoints += mapPoint.getNDatas();
dataBlock.loadViewerData(mapPoint.getStartTime(), mapPoint.getEndTime(), null);
ArrayList<PamDataUnit> dataCopy = dataBlock.getDataCopy(deployment.getAudioStart(), deployment.getAudioEnd(), true, dataSelector);
// System.out.printf("%d loaded from %s to %s %d kept\n", dataBlock.getUnitsCount(), PamCalendar.formatDateTime(mapPoint.getStartTime()),
// PamCalendar.formatDateTime(mapPoint.getEndTime()), dataCopy.size());
totalLoadedDatas += dataCopy.size();
System.out.printf("%d loaded from %s to %s %d kept\n", dataBlock.getUnitsCount(), PamCalendar.formatDateTime(mapPoint.getStartTime()),
PamCalendar.formatDateTime(mapPoint.getEndTime()), dataCopy.size());
skipCount += dataBlock.getUnitsCount() - dataCopy.size();
for (PamDataUnit dataUnit : dataCopy) {
/*
@ -334,7 +347,7 @@ public class DetectionsHandler {
documentCount+=dets.length;
if (exportCount % 100 == 0) {
prog = new DetectionExportProgress(deployment, null,
prog = new DetectionExportProgress(deployment, null,totalMapPoints, doneMapPoints,
lastUnitTime, totalCount, exportCount, skipCount, DetectionExportProgress.STATE_COUNTING);
exportObserver.update(prog);
}
@ -345,15 +358,17 @@ public class DetectionsHandler {
// onEffort.getDetection().add(det);
lastUnitTime = dataUnit.getTimeMilliseconds();
}
prog = new DetectionExportProgress(deployment, null,
doneMapPoints++;
prog = new DetectionExportProgress(deployment, null,totalMapPoints, doneMapPoints,
lastUnitTime, totalCount, exportCount, skipCount, DetectionExportProgress.STATE_COUNTING);
exportObserver.update(prog);
if (viewerLoadPolicy == ViewerLoadPolicy.LOAD_ALWAYS_EVERYTHING) {
break;
}
if (!activeExport) {
return 0;
}
}
Detection dets[] = granularityHandler.cleanup(deployment.getAudioEnd());
if (dets != null) {
@ -397,28 +412,33 @@ public class DetectionsHandler {
if (viewerLoadPolicy == null) {
viewerLoadPolicy = ViewerLoadPolicy.LOAD_UTCNORMAL;
}
int totalMapPoints = dataMap.getNumMapPoints();
int doneMapPoints = 0;
GranularityHandler granularityHandler = GranularityHandler.getHandler(streamExportParams.granularity, tethysControl, dataBlock, exportParams, streamExportParams);
for (PDeployment deployment : deployments) {
int documentCount = 0;
prog = new DetectionExportProgress(deployment, null,
prog = new DetectionExportProgress(deployment, null,totalMapPoints, doneMapPoints,
lastUnitTime, totalCount, exportCount, skipCount, DetectionExportProgress.STATE_COUNTING);
exportObserver.update(prog);
granularityHandler.prepare(deployment.getAudioStart());
if (currentDetections == null) {
currentDetections = startDetectionsDocument(deployment, dataBlock, streamExportParams);
currentDetections.getEffort().setStart(TethysTimeFuncs.xmlGregCalFromMillis(deployment.getAudioStart()));
}
// export everything in that deployment.
// need to loop through all map points in this interval.
List<OfflineDataMapPoint> mapPoints = dataMap.getMapPoints();
for (OfflineDataMapPoint mapPoint : mapPoints) {
if (!activeExport) {
prog = new DetectionExportProgress(deployment, currentDetections,
prog = new DetectionExportProgress(deployment, currentDetections,totalMapPoints, doneMapPoints,
lastUnitTime, totalCount, exportCount, skipCount, DetectionExportProgress.STATE_CANCELED);
exportObserver.update(prog);
break;
}
if (currentDetections == null) {
// needed in inner loop in case doc gets written at 500000.
currentDetections = startDetectionsDocument(deployment, dataBlock, streamExportParams);
currentDetections.getEffort().setStart(TethysTimeFuncs.xmlGregCalFromMillis(deployment.getAudioStart()));
}
if (mapPoint.getEndTime() < deployment.getAudioStart()) {
continue;
}
@ -427,6 +447,7 @@ public class DetectionsHandler {
}
dataBlock.loadViewerData(mapPoint.getStartTime(), mapPoint.getEndTime(), null);
ArrayList<PamDataUnit> dataCopy = dataBlock.getDataCopy(deployment.getAudioStart(), deployment.getAudioEnd(), true, dataSelector);
Collections.sort(dataCopy);
skipCount += dataBlock.getUnitsCount() - dataCopy.size();
DetectionGroup onEffort = currentDetections.getOnEffort();
for (PamDataUnit dataUnit : dataCopy) {
@ -442,24 +463,27 @@ public class DetectionsHandler {
}
}
if (exportCount % 100 == 0) {
prog = new DetectionExportProgress(deployment, null,
prog = new DetectionExportProgress(deployment, currentDetections, totalMapPoints, doneMapPoints,
lastUnitTime, totalCount, exportCount, skipCount, DetectionExportProgress.STATE_GATHERING);
exportObserver.update(prog);
}
lastUnitTime = dataUnit.getTimeMilliseconds();
}
prog = new DetectionExportProgress(deployment, currentDetections,
doneMapPoints ++;
prog = new DetectionExportProgress(deployment, currentDetections,totalMapPoints, doneMapPoints,
lastUnitTime, totalCount, exportCount, skipCount, DetectionExportProgress.STATE_GATHERING);
exportObserver.update(prog);
if (documentCount > 500000 && mapPoint != dataMap.getLastMapPoint()) {
prog = new DetectionExportProgress(deployment, currentDetections,
if (documentCount > 50000000 && mapPoint != dataMap.getLastMapPoint()) {
prog = new DetectionExportProgress(deployment, currentDetections,totalMapPoints, doneMapPoints,
lastUnitTime, totalCount, exportCount, skipCount, DetectionExportProgress.STATE_WRITING);
exportObserver.update(prog);
closeDetectionsDocument(currentDetections, mapPoint.getEndTime());
try {
dbxmlConnect.postAndLog(currentDetections);
if (checkDetectionsDocument(currentDetections, granularityHandler)) {
dbxmlConnect.postAndLog(currentDetections);
}
} catch (TethysException e) {
tethysControl.showException(e);
}
@ -469,8 +493,14 @@ public class DetectionsHandler {
if (viewerLoadPolicy == ViewerLoadPolicy.LOAD_ALWAYS_EVERYTHING) {
break;
}
if (!activeExport) {
break;
}
}
if (!activeExport) {
return DetectionExportProgress.STATE_CANCELED;
}
if (currentDetections != null) {
Detection dets[] = granularityHandler.cleanup(deployment.getAudioEnd());
@ -481,11 +511,13 @@ public class DetectionsHandler {
currentDetections.getOnEffort().getDetection().add(dets[dd]);
}
}
prog = new DetectionExportProgress(deployment, currentDetections,
prog = new DetectionExportProgress(deployment, currentDetections,totalMapPoints, doneMapPoints,
lastUnitTime, totalCount, exportCount, skipCount, DetectionExportProgress.STATE_WRITING);
closeDetectionsDocument(currentDetections, deployment.getAudioEnd());
try {
dbxmlConnect.postAndLog(currentDetections);
if (checkDetectionsDocument(currentDetections, granularityHandler)) {
dbxmlConnect.postAndLog(currentDetections);
}
} catch (TethysException e) {
tethysControl.showException(e);
}
@ -493,7 +525,7 @@ public class DetectionsHandler {
}
}
prog = new DetectionExportProgress(null, null,
prog = new DetectionExportProgress(null, null,totalMapPoints, totalMapPoints,
lastUnitTime, totalCount, exportCount, skipCount, DetectionExportProgress.STATE_COMPLETE);
exportObserver.update(prog);
return DetectionExportProgress.STATE_COMPLETE;
@ -552,7 +584,7 @@ public class DetectionsHandler {
supportSoft.setVersion(getSupportSoftwareVersion(dataBlock));
supSoft.add(supportSoft);
detections.setAlgorithm(algorithm);
detections.setUserId("Unknown user");
detections.setUserId("PAMGuard user");
detections.setEffort(getDetectorEffort(deployment, dataBlock, exportParams));
return detections;
@ -567,6 +599,50 @@ public class DetectionsHandler {
private void closeDetectionsDocument(Detections detections, Long audioEnd) {
detections.getEffort().setEnd(TethysTimeFuncs.xmlGregCalFromMillis(audioEnd));
}
/**
* Run some checks on the Detections document prior to submission. <br>
* Currently, is is just a check that the detections are within the effort times.
* @param detections Detections document
* @return false if there is an outstanding problem.
*/
private boolean checkDetectionsDocument(Detections detections, GranularityHandler granularityHandler) {
XMLGregorianCalendar effStart = detections.getEffort().getStart();
XMLGregorianCalendar effEnd = detections.getEffort().getEnd();
DetectionGroup dets = detections.getOnEffort();
List<Detection> detList = dets.getDetection();
ListIterator<Detection> detIt = detList.listIterator();
while (detIt.hasNext()) {
Detection det = detIt.next();
XMLGregorianCalendar detS = det.getStart();
XMLGregorianCalendar detE = det.getEnd();
if (effStart.compare(detS) == DatatypeConstants.GREATER) {
if (granularityHandler.autoEffortFix(detections, det)) {
continue;
}
String str = String.format("<html>A Detection at %s starts before the document effort start at %s<br>"
+ "Do you want to adjust the effort start time or abort export ?</html>", detS, effStart);
int ans = WarnOnce.showNamedWarning("TETHYSDETNOTINEFFORT", tethysControl.getGuiFrame(), "Detection Document Warning", str, WarnOnce.OK_CANCEL_OPTION);
if (ans == WarnOnce.CANCEL_OPTION) {
return false;
}
detections.getEffort().setStart(detS);
}
if (effEnd.compare(detE) == DatatypeConstants.LESSER) {
if (granularityHandler.autoEffortFix(detections, det)) {
continue;
}
String str = String.format("<html>A Detection at %s-%s ends <br>after the document effort end at %s<br>"
+ "Do you want to adjust the effort end time or abort export ?</html>", detS, detE, effStart);
int ans = WarnOnce.showNamedWarning("TETHYSDETNOTINEFFORT", tethysControl.getGuiFrame(), "Detection Document Warning", str, WarnOnce.OK_CANCEL_OPTION);
if (ans == WarnOnce.CANCEL_OPTION) {
return false;
}
detections.getEffort().setEnd(detE);
}
}
return true;
}
/**
* Worker thread for exporting detections.
@ -599,13 +675,16 @@ public class DetectionsHandler {
protected Integer doInBackground() throws Exception {
Integer ans = null;
try {
int count = countDetections(dataBlock, exportParams, exportObserver);
String msg = String.format("Do you want to go ahead and output %d %s detections to Tethys?",
count, exportParams.granularity);
int doit = WarnOnce.showWarning("Tethys Detections Export", msg, WarnOnce.OK_CANCEL_OPTION);
if (doit == WarnOnce.OK_OPTION) {
// int count = countDetections(dataBlock, exportParams, exportObserver);
// if (activeExport == false) {
// return 0;
// }
// String msg = String.format("Do you want to go ahead and output %d %s detections to Tethys?",
// count, exportParams.granularity);
// int doit = WarnOnce.showWarning("Tethys Detections Export", msg, WarnOnce.OK_CANCEL_OPTION);
// if (doit == WarnOnce.OK_OPTION) {
ans = exportDetections(dataBlock, exportParams, this);
}
// }
}
catch (Exception e) {
e.printStackTrace();
@ -616,7 +695,7 @@ public class DetectionsHandler {
@Override
protected void done() {
// this.
DetectionExportProgress prog = new DetectionExportProgress(null, null, 0, 0, 0, 0, DetectionExportProgress.STATE_COMPLETE);
DetectionExportProgress prog = new DetectionExportProgress(null, null, 0, 0, 0, 0, 0, 0, DetectionExportProgress.STATE_COMPLETE);
tethysControl.exportedDetections(dataBlock);
exportObserver.update(prog);
TethysReporter.getTethysReporter().showReport(tethysControl.getGuiFrame(), true);
@ -661,4 +740,10 @@ public class DetectionsHandler {
DetectionsExportWizard.showDialog(tethysControl.getGuiFrame(), tethysControl, dataBlock);
}
@Override
public String getHelpPoint() {
return helpPoint;
}
}

35
src/tethys/detection/EncounterGranularityHandler.java
View File

@ -11,6 +11,7 @@ import java.util.Map.Entry;
import PamguardMVC.PamDataBlock;
import PamguardMVC.PamDataUnit;
import nilus.Detection;
import nilus.Detections;
import nilus.SpeciesIDType;
import tethys.TethysControl;
import tethys.TethysTimeFuncs;
@ -48,7 +49,7 @@ public class EncounterGranularityHandler extends GranularityHandler {
@Override
public void prepare(long timeMillis) {
currentDetections.clear();
}
@Override
@ -77,6 +78,7 @@ public class EncounterGranularityHandler extends GranularityHandler {
currentDetections.put(groupName, det);
}
else {
// add to current detection. Set new end time and increment count
det.setEnd(TethysTimeFuncs.xmlGregCalFromMillis(dataUnit.getEndTimeInMilliseconds()));
int count = det.getCount().intValue() + 1;
@ -122,34 +124,17 @@ public class EncounterGranularityHandler extends GranularityHandler {
}
}
// private Detection[] checkCurrentEncounters(long timeMilliseconds) {
// if (currentDetections == null || currentDetections.size() == 0) {
// return null;
// }
// int nGood = 0;
// Detection[] newDetections = new Detection[currentDetections.size()];
// Iterator<Detection> detIt = currentDetections.iterator();
// while (detIt.hasNext()) {
// Detection aDet = detIt.next();
// Long detEnd = TethysTimeFuncs.millisFromGregorianXML(aDet.getEnd());
// if (timeMilliseconds-detEnd > maxGapMillis) {
// detIt.remove();
// newDetections[nGood++] = aDet;
// }
// }
//
// if (nGood == 0) {
// return null;
// }
// else {
// return Arrays.copyOf(newDetections, nGood);
// }
// }
@Override
public Detection[] cleanup(long timeMillis) {
// get everything still on the go.
return checkCurrentEncounters(timeMillis + maxGapMillis);
return checkCurrentEncounters(timeMillis + maxGapMillis*10);
}
@Override
protected boolean autoEffortFix(Detections detections, Detection det) {
return expandEffort(detections, det);
}
}

97
src/tethys/detection/GranularityHandler.java
View File

@ -1,8 +1,16 @@
package tethys.detection;
import java.util.List;
import java.util.ListIterator;
import javax.xml.datatype.DatatypeConstants;
import javax.xml.datatype.XMLGregorianCalendar;
import PamguardMVC.PamDataBlock;
import PamguardMVC.PamDataUnit;
import nilus.Detection;
import nilus.DetectionGroup;
import nilus.Detections;
import nilus.GranularityEnumType;
import tethys.TethysControl;
import tethys.output.StreamExportParams;
@ -118,4 +126,93 @@ public abstract class GranularityHandler {
}
return null;
}
/**
* Automatically fix mismatches between effort and detections. This will be called if a
* detection or part of a detection is outside of the start and end defined by the effort. If it's a
* small difference, i.e. if the detection at least overlaps the effort then it can be automatically
* fixed by truncating the detection (for binned types) or by a small extension to the effort (for encounter
* and call types).
* @param detections nilus Detections object
* @param det a single detection
* @return true if it was fixed automatically. False otherwise.
*/
protected abstract boolean autoEffortFix(Detections detections, Detection det);
/**
* Check that the detection at least overlaps the effort period.
* @param detections nilus Detections object
* @param det a single detection
* @return true if the overlap
*/
protected boolean effortOverlap(Detections detections, Detection det) {
XMLGregorianCalendar effStart = detections.getEffort().getStart();
XMLGregorianCalendar effEnd = detections.getEffort().getEnd();
XMLGregorianCalendar detStart = det.getStart();
XMLGregorianCalendar detEnd = det.getEnd();
if (effStart.compare(detEnd) == DatatypeConstants.GREATER) {
return false;
}
if (effEnd.compare(detStart) == DatatypeConstants.LESSER) {
return false;
}
return true;
}
/**
* Fix effort / detection problem but contracting the start / end times of the detection
* @param detections nilus Detections object
* @param det a single detection
* @return true if fixed automatically
*/
protected boolean contractDetection(Detections detections, Detection det) {
if (effortOverlap(detections, det) == false) {
return false;
}
// at least some overlap, so fix it.
// going to fix it my shortening the detection, and leave the effort alone.
XMLGregorianCalendar effStart = detections.getEffort().getStart();
XMLGregorianCalendar effEnd = detections.getEffort().getEnd();
XMLGregorianCalendar detStart = det.getStart();
XMLGregorianCalendar detEnd = det.getEnd();
if (effStart.compare(detStart) == DatatypeConstants.GREATER) {
System.out.printf("Fix Detections change detection start from %s to %s\n", detStart, effStart);
det.setStart(effStart);
}
if (effEnd.compare(detEnd) == DatatypeConstants.LESSER) {
System.out.printf("Fix Detections change detection end from %s to %s\n", detEnd, effEnd);
det.setEnd(effEnd);
}
return true;
}
/**
* Fix effort / detection problem but expanding the start / end times of the effort
* @param detections nilus Detections object
* @param det a single detection
* @return true if fixed automatically
*/
protected boolean expandEffort(Detections detections, Detection det) {
if (effortOverlap(detections, det) == false) {
return false;
}
// at least some overlap, so fix it.
// going to fix it my shortening the detection, and leave the effort alone.
XMLGregorianCalendar effStart = detections.getEffort().getStart();
XMLGregorianCalendar effEnd = detections.getEffort().getEnd();
XMLGregorianCalendar detStart = det.getStart();
XMLGregorianCalendar detEnd = det.getEnd();
if (effStart.compare(detStart) == DatatypeConstants.GREATER) {
System.out.printf("Fix Detections change effort start from %s to %s\n", effStart, detStart);
detections.getEffort().setStart(detStart);
}
if (effEnd.compare(detEnd) == DatatypeConstants.LESSER) {
System.out.printf("Fix Detections change effort end from %s to %s\n", effEnd, detEnd);
detections.getEffort().setEnd(detEnd);
}
return true;
}
}

Some files were not shown because too many files have changed in this diff Show More