From b08e86e330c5ca2de4b6902eb7ec514f47278a7c Mon Sep 17 00:00:00 2001 From: Jamie Mac Date: Wed, 17 Aug 2022 09:16:52 +0100 Subject: [PATCH] Bug fix for click train detector (#49) * Updates to click train detector New GUI for click train classification - more intuitive and allows users to build more powerful classifiers. CPOD data can now build average waveforms. CPOD click trains can be viewed in TDisplayFX pop up menu displays Fixed Peak Frequency symbol chooser so it saves the colour box settings * Create click_train_help.md * Update click_train_help.md * Screenshots * Update click_train_help.md * Update click_train_help.md * Update click_train_help.md * More screenshots * Add screenshots * Update click_train_help.md * Add classifier screenshot * Update click_train_help.md * Update click_train_help.md * Update click_train_help.md * Updates and bug fixes to click train detector and CPOD importer * Update click_train_help.md * Update POM with latest jdl4pam * Add screenshots for click train detector help * Screenshots * Update click_train_help.md * Defult option for CPOD and porpoise to click train detector. Also a minor * Update pom.xml * Bug fix for raw spectrogram in TDDisplayFX * Update KetosClassifier.java * Updates to click train detector New GUI for click train classification - more intuitive and allows users to build more powerful classifiers. CPOD data can now build average waveforms. CPOD click trains can be viewed in TDisplayFX pop up menu displays Fixed Peak Frequency symbol chooser so it saves the colour box settings * Create click_train_help.md * Update click_train_help.md * Screenshots * Update click_train_help.md * More screenshots * Update click_train_help.md * Update click_train_help.md * Add screenshots * Add classifier screenshot * Update click_train_help.md * Updates and bug fixes to click train detector and CPOD importer * Update click_train_help.md * Update click_train_help.md * Update click_train_help.md * Update click_train_help.md * Add screenshots for click train detector help * Screenshots * Update click_train_help.md * Defult option for CPOD and porpoise to click train detector. Also a minor * Bug fix for raw spectrogram in TDDisplayFX * Update KetosClassifier.java * Fix standard classifier JSON logging * Update click_train_help.md * Fixed bug in sweep classifier when using SoundTrap click detections * Added some colour averaging in the TFDisplayFX spectrgoram * Bug fix for rainbow click bearings Bug fix when rainbow clicks are imported mean bearings cannot be calculated - was an array size issue in DelayGroup * Google humpback whale deep learning classifier Google's humpback whale deep learning classifier can now be imported. Updated TDisplayFX to make the data selection panes cleaner and clearer. Updated the TDisplayFX so that predicitons from deep learning models now have some display options e.g. changing colour. * Updated the prediction plots on time display Deep learning prediciton plots on the time display have now been updated to have some colour options and also act as a legend. * Updates to TDisplayFX yFIshmael now owrks with TDisplayFX TDisplayFX UI changes to make simpler. Some abstraction for drawing lines on TDisplayFX * Updates to click train detector New GUI for click train classification - more intuitive and allows users to build more powerful classifiers. CPOD data can now build average waveforms. CPOD click trains can be viewed in TDisplayFX pop up menu displays Fixed Peak Frequency symbol chooser so it saves the colour box settings * Create click_train_help.md * Update click_train_help.md * Screenshots * Update click_train_help.md * More screenshots * Update click_train_help.md * Update click_train_help.md * Add screenshots * Add classifier screenshot * Update click_train_help.md * Updates and bug fixes to click train detector and CPOD importer * Update click_train_help.md * Update click_train_help.md * Update click_train_help.md * Update click_train_help.md * Add screenshots for click train detector help * Screenshots * Update click_train_help.md * Defult option for CPOD and porpoise to click train detector. Also a minor * Updates to click train detector New GUI for click train classification - more intuitive and allows users to build more powerful classifiers. CPOD data can now build average waveforms. CPOD click trains can be viewed in TDisplayFX pop up menu displays Fixed Peak Frequency symbol chooser so it saves the colour box settings * Create click_train_help.md * Update click_train_help.md * Update click_train_help.md * Updates and bug fixes to click train detector and CPOD importer * Bug fix for raw spectrogram in TDDisplayFX * Update KetosClassifier.java * Fix standard classifier JSON logging * Fixed bug in sweep classifier when using SoundTrap click detections * Added some colour averaging in the TFDisplayFX spectrgoram * Bug fix for rainbow click bearings Bug fix when rainbow clicks are imported mean bearings cannot be calculated - was an array size issue in DelayGroup * Google humpback whale deep learning classifier Google's humpback whale deep learning classifier can now be imported. Updated TDisplayFX to make the data selection panes cleaner and clearer. Updated the TDisplayFX so that predicitons from deep learning models now have some display options e.g. changing colour. * Updated the prediction plots on time display Deep learning prediciton plots on the time display have now been updated to have some colour options and also act as a legend. * Updates to TDisplayFX yFIshmael now owrks with TDisplayFX TDisplayFX UI changes to make simpler. Some abstraction for drawing lines on TDisplayFX * Merge fixes to click train detector * Bug fix to UI * Updates to FX GUI * Updates to click train detector New GUI for click train classification - more intuitive and allows users to build more powerful classifiers. CPOD data can now build average waveforms. CPOD click trains can be viewed in TDisplayFX pop up menu displays Fixed Peak Frequency symbol chooser so it saves the colour box settings * Screenshots * Updates and bug fixes to click train detector and CPOD importer * Updates to click train detector New GUI for click train classification - more intuitive and allows users to build more powerful classifiers. CPOD data can now build average waveforms. CPOD click trains can be viewed in TDisplayFX pop up menu displays Fixed Peak Frequency symbol chooser so it saves the colour box settings * Updates and bug fixes to click train detector and CPOD importer * Fix standard classifier JSON logging * Merge fixes to click train detector * Updates to click train detector New GUI for click train classification - more intuitive and allows users to build more powerful classifiers. CPOD data can now build average waveforms. CPOD click trains can be viewed in TDisplayFX pop up menu displays Fixed Peak Frequency symbol chooser so it saves the colour box settings * Screenshots * Updates and bug fixes to click train detector and CPOD importer * Updates to click train detector New GUI for click train classification - more intuitive and allows users to build more powerful classifiers. CPOD data can now build average waveforms. CPOD click trains can be viewed in TDisplayFX pop up menu displays Fixed Peak Frequency symbol chooser so it saves the colour box settings * Screenshots * Updates and bug fixes to click train detector and CPOD importer * Fix standard classifier JSON logging * Google humpback whale deep learning classifier Google's humpback whale deep learning classifier can now be imported. Updated TDisplayFX to make the data selection panes cleaner and clearer. Updated the TDisplayFX so that predicitons from deep learning models now have some display options e.g. changing colour. * Bug fix to UI * Bug fixes to FX GUI * Updates to click train detector * Squashed commit of the following: commit 9f998165ee95dbb16acaf420876f3aeab75c2158 Author: Douglas Gillespie <50671166+douggillespie@users.noreply.github.com> Date: Mon May 2 19:40:24 2022 +0100 Updates to support ContactCollator plugin (#33) * Change synchronization on RawDataTransforms soit uses the owner data holder, not itself for synchronization. Otherwise you get thread locks. * fix problem in SummaryComand * Update command line options * Change synchronization on RawDataTransforms soit uses the owner data holder, not itself for synchronization. Otherwise you get thread locks. * Update command line options * Update DecimatorParams.java * couple of updates to support new contact collator plugin * Sorting out sample rate info in clip display to support Contact Collator plugin * FLAC Speed Improve flac speed * Update .gitignore * Update .gitignore * Updates to support new features in Contact Collator * Small update to RawDatautils to handle null data * Updates to click train detector * Squashed commit of the following: commit 62b020b3204aa56189b1c2da88bcbb9f49140936 Author: Douglas Gillespie <50671166+douggillespie@users.noreply.github.com> Date: Sat May 14 06:52:20 2022 +0100 Add a new offlinefileslist function commit 3a9a5311aa529b66340f6ae322f88905b911947a Author: Douglas Gillespie <50671166+douggillespie@users.noreply.github.com> Date: Wed Apr 27 09:43:31 2022 +0100 Update .gitignore commit 9f998165ee95dbb16acaf420876f3aeab75c2158 Author: Douglas Gillespie <50671166+douggillespie@users.noreply.github.com> Date: Mon May 2 19:40:24 2022 +0100 Updates to support ContactCollator plugin (#33) * Change synchronization on RawDataTransforms soit uses the owner data holder, not itself for synchronization. Otherwise you get thread locks. * fix problem in SummaryComand * Update command line options * Change synchronization on RawDataTransforms soit uses the owner data holder, not itself for synchronization. Otherwise you get thread locks. * Update command line options * Update DecimatorParams.java * couple of updates to support new contact collator plugin * Sorting out sample rate info in clip display to support Contact Collator plugin * FLAC Speed Improve flac speed * Update .gitignore * Update .gitignore * Updates to support new features in Contact Collator * Small update to RawDatautils to handle null data * Updated data selector for click train detector * Click train detector updates and bug fixes Fixed very annoying bug which meant templates did not show properly when the dialog was first opened in swing. Added feature to simple classifier were % of clicks of one click classification (or other data selector paramter) can be used to classify. * Bug fix and improvements for Ketos Ketos models now automatically set the correct sample length when loaded. Bug fix for the TDisplayFX - null pointer exception if no class names loaded and sorted recently introduced bug in drawing predictions. * Updates to deep learning and FX GUI Added some more example sounds to deep learning UI. Updated click detector in the FX GUI. * PAMGuard FX sound output update * Updates to FX GUI * Updates to FX GUI * Update RawDataTransforms.java * Fixes to FX UI * Squashed commit of the following: commit 11ba8bf91e1cd55e4305c4060a379f9ff0c58c4d Author: Douglas Gillespie <50671166+douggillespie@users.noreply.github.com> Date: Thu Aug 4 11:45:47 2022 +0100 MErge from DG Branch (#47) * Variable sound output level Mods to SoundPlayback module to allow additional parameters. Implemented system for NI cards to allow changes to selected output voltage range meaning output can be boosted to level higher than current default. * Code to support nogui operations when no screens present on headless system * Fix problem of nogui headless operation trying to access screen size. * Click detector display fixes 1. ICI not displaying correctly 2. Component sizes in display dialog on hres monitors * Work on batch processing, after testing of options to autostart, autoexit and set wav file folder, database and binary store. * Update MHTClickTrainAlgorithm.java Fix unsynchronised access to a datablock in click train detector which was causing index errors. * Revamp of offline process messaging and control Includes some databsae logging of completed tasks * Offline task logging Bit more work, including notes and database storage of task reprocessing. Guess this could all become 'proper' PAMGuard data and be shown in a table on the display but that not priority enough. * Dialog packing Fix a couple of dialogs which don't back well on HDPI monitors * UDP Control Added multiport functionality commit 9a9f542d95b6cbc571d60a2250a2d4accf8dc4b3 Author: Douglas Gillespie <50671166+douggillespie@users.noreply.github.com> Date: Thu Aug 4 11:42:45 2022 +0100 Merge DG to Main (#46) * Variable sound output level Mods to SoundPlayback module to allow additional parameters. Implemented system for NI cards to allow changes to selected output voltage range meaning output can be boosted to level higher than current default. * Code to support nogui operations when no screens present on headless system * Fix problem of nogui headless operation trying to access screen size. * Click detector display fixes 1. ICI not displaying correctly 2. Component sizes in display dialog on hres monitors * Work on batch processing, after testing of options to autostart, autoexit and set wav file folder, database and binary store. * Update MHTClickTrainAlgorithm.java Fix unsynchronised access to a datablock in click train detector which was causing index errors. * Revamp of offline process messaging and control Includes some databsae logging of completed tasks * Offline task logging Bit more work, including notes and database storage of task reprocessing. Guess this could all become 'proper' PAMGuard data and be shown in a table on the display but that not priority enough. * Dialog packing Fix a couple of dialogs which don't back well on HDPI monitors * UDP Control Added multiport functionality commit 49cd547aee3e4366e1aaa51b8fe7418199c2ad73 Author: Douglas Gillespie <50671166+douggillespie@users.noreply.github.com> Date: Thu Aug 4 11:40:29 2022 +0100 Merge DG branch (#45) * Variable sound output level Mods to SoundPlayback module to allow additional parameters. Implemented system for NI cards to allow changes to selected output voltage range meaning output can be boosted to level higher than current default. * Code to support nogui operations when no screens present on headless system * Fix problem of nogui headless operation trying to access screen size. * Click detector display fixes 1. ICI not displaying correctly 2. Component sizes in display dialog on hres monitors * Work on batch processing, after testing of options to autostart, autoexit and set wav file folder, database and binary store. * Update MHTClickTrainAlgorithm.java Fix unsynchronised access to a datablock in click train detector which was causing index errors. * Revamp of offline process messaging and control Includes some databsae logging of completed tasks * Offline task logging Bit more work, including notes and database storage of task reprocessing. Guess this could all become 'proper' PAMGuard data and be shown in a table on the display but that not priority enough. * Dialog packing Fix a couple of dialogs which don't back well on HDPI monitors * UDP Control Added multiport functionality commit 016cfd0da54021dcbe787f0a20856604b81bf7af Author: Douglas Gillespie <50671166+douggillespie@users.noreply.github.com> Date: Thu Aug 4 11:35:06 2022 +0100 Dialog positioning New functions to better positions dialogs on screen commit c9f2ab3e97ec289ee8c1bcfe4531c84f77ee7fa8 Author: Douglas Gillespie <50671166+douggillespie@users.noreply.github.com> Date: Mon Aug 1 11:13:24 2022 +0100 puch to main (#44) * Variable sound output level Mods to SoundPlayback module to allow additional parameters. Implemented system for NI cards to allow changes to selected output voltage range meaning output can be boosted to level higher than current default. * Code to support nogui operations when no screens present on headless system * Fix problem of nogui headless operation trying to access screen size. * Click detector display fixes 1. ICI not displaying correctly 2. Component sizes in display dialog on hres monitors * Work on batch processing, after testing of options to autostart, autoexit and set wav file folder, database and binary store. * Update MHTClickTrainAlgorithm.java Fix unsynchronised access to a datablock in click train detector which was causing index errors. * Revamp of offline process messaging and control Includes some databsae logging of completed tasks * Offline task logging Bit more work, including notes and database storage of task reprocessing. Guess this could all become 'proper' PAMGuard data and be shown in a table on the display but that not priority enough. * Dialog packing Fix a couple of dialogs which don't back well on HDPI monitors commit 55f5a3fcf12c6c8a727318a17f9dece401adc9f3 Author: Douglas Gillespie <50671166+douggillespie@users.noreply.github.com> Date: Mon Aug 1 10:59:36 2022 +0100 Group detections menu Small changes to limit the number of menu items in "Add to existing group" to a maximum of 25 entries. commit b3f6c0e6657de91a7e015a48f5f218f60f23c5f5 Author: Douglas Gillespie <50671166+douggillespie@users.noreply.github.com> Date: Fri Jul 29 10:50:45 2022 +0100 Handle -nogui option in PamWorker PamWorker used to catalog files at startup (if a file folder input system is used). This creates a progress dialog. Stop it appearing in -nogui operations. commit 8569b6b579659cba8229596e91c35355bc5ed61b Author: Douglas Gillespie <50671166+douggillespie@users.noreply.github.com> Date: Fri Jul 22 08:30:47 2022 +0100 Click display fixes (#41) * Variable sound output level Mods to SoundPlayback module to allow additional parameters. Implemented system for NI cards to allow changes to selected output voltage range meaning output can be boosted to level higher than current default. * Code to support nogui operations when no screens present on headless system * Fix problem of nogui headless operation trying to access screen size. * Click detector display fixes 1. ICI not displaying correctly 2. Component sizes in display dialog on hres monitors commit 128a512ff60c26b5acedd76de96c0905c6bf27ae Author: Douglas Gillespie <50671166+douggillespie@users.noreply.github.com> Date: Tue Jul 19 16:37:57 2022 +0100 Another attempt at stopping it crashing on a headless system Dealing with displays that get created for clickangle vetos and a call to the gui in the click train detector. commit 6eaa6e4978b2483e5828ca4a3914c58d7a9cfe2d Author: Douglas Gillespie <50671166+douggillespie@users.noreply.github.com> Date: Tue Jul 19 08:20:07 2022 +0100 nogoi fix for headless systems. (#40) * Variable sound output level Mods to SoundPlayback module to allow additional parameters. Implemented system for NI cards to allow changes to selected output voltage range meaning output can be boosted to level higher than current default. * Code to support nogui operations when no screens present on headless system * Fix problem of nogui headless operation trying to access screen size. commit 9fdd30556bf391428e1d3f2f9e91dbbdf771fa84 Author: Douglas Gillespie <50671166+douggillespie@users.noreply.github.com> Date: Tue Jul 12 15:53:07 2022 +0100 Variable sound output level (#39) Mods to SoundPlayback module to allow additional parameters. Implemented system for NI cards to allow changes to selected output voltage range meaning output can be boosted to level higher than current default. * Bug fixes to click train detector Fixed bug with number of coasts in the MHT Kernel Added an option for amplitude to have a maximum Removed print statements. * Bug fix for click train logging The size of the string classifers field was dynamically set - this was a bug. Co-authored-by: Douglas Gillespie <50671166+douggillespie@users.noreply.github.com> --- .../logging/ClickTrainDetLogging.java | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/src/clickTrainDetector/logging/ClickTrainDetLogging.java b/src/clickTrainDetector/logging/ClickTrainDetLogging.java index 4ebe115d..ad537e53 100644 --- a/src/clickTrainDetector/logging/ClickTrainDetLogging.java +++ b/src/clickTrainDetector/logging/ClickTrainDetLogging.java @@ -133,12 +133,15 @@ public class ClickTrainDetLogging extends SuperDetLogging { String classificationData = getClassifierData(ctDataUnit); - // Debug.out.println("Classifier Save: " + classificationData); + System.out.println("Classifier Save: " + classificationData); if (classificationData.length()>0) { - classifiers.setLength(classificationData.length()); + //classifiers.setLength(classificationData.length()); classifiers.setValue(classificationData); } + else { + classifiers.setValue(null); + } if (ctDataUnit.getClassificationIndex()>=0) { speciesFlag.setValue(ctDataUnit.getCtClassifications(). @@ -301,8 +304,8 @@ public class ClickTrainDetLogging extends SuperDetLogging { } } catch (Exception e) { - Debug.err.println("ClickTrainDetLogging: BAD JSON CLASSIFIER STRING: " + classifiersDatas[i]); - Debug.err.println(classifiersData); + System.err.println("ClickTrainDetLogging: BAD JSON CLASSIFIER STRING: " + classifiersDatas[i]); + System.err.println(classifiersData); e.printStackTrace(); } }