+name="_Latest_Version_2.02.06">Latest Version 2.02.08 May
+2023
Bug Fixes
-
Use of localization sensor and orientation data
-for static hydrophones had a bug whereby it would continually ‘forget’ angle
-offsets applied to static hydrophones in viewer mode. This is now fixed.
+
ROCCA Memory Leak: A memory leak in ROCCA,
+which mostly occurred when processing large datasets of many offline files, has
+been fixed.
+
+
New Features
+
+
Restart Options: When you restart
+processing of offline files, if output data already exist (binary data of
+within database tables) you will be asked if you want to overwrite the data, cancel,
+or try to continue from where to left off.
+
+
Updated SoundTrap sud file interface so
+that PAMGuard now extracts and stores the Click Detector settings from the sud
+files and stores them within the PAMGuard configuration. This does not affect
+processing, but is important for record keeping.
+
+
Version 2.02.07 January 2023
+
+
Bug Fixes
+
+
Use of localization sensor and orientation
+data for static hydrophones had a bug whereby it would continually ‘forget’
+angle offsets applied to static hydrophones in viewer mode. This is now fixed.
Click tool bar: Correctly shows event
selection options even if no species classification options are in place.
@@ -467,8 +491,8 @@ selection options even if no species classification options are in place.
ROCCA: Fixed (another) memory leak which
-caused PAMGuard to crash when processing large data sets with the ROCCA
-classifier.
+caused PAMGuard to crash when processing large data sets with the ROCCA classifier.
+
Ishmael Modules: Fixed bug which
occasionally caused crashes when processing many files offline.
10. Bug 442. BT Display does not change click symbol color when Colour
-By Superdetection is selected.
+lang=EN-US> Bug 442. BT Display does not change click symbol color when
+Colour By Superdetection is selected.
11.
15.
-Bug 447. Viewer mode throws exception when trying to load beamformer localisations
+Bug 447. Viewer mode throws exception when trying to load beamformer
+localisations
16.
18. Bug 450. Ishmael Detectors subscribing to FFTDataBlock twice,
-meaning they process each data unit 2x doubling the output.
+lang=EN-US> Bug 450. Ishmael Detectors subscribing to FFTDataBlock
+twice, meaning they process each data unit 2x doubling the output.
Upgrades
@@ -1596,9 +1621,9 @@ lang=EN-US> Added decimal degrees option to latitude/longitude dialog.2. Added option in the Click Delay parameters to restrict the number
-of initial samples used in the calculation. This can be useful for cleaning up
-delays in the presence of echoes and small time delays.
+lang=EN-US> Added option in the Click Delay parameters to restrict the
+number of initial samples used in the calculation. This can be useful for
+cleaning up delays in the presence of echoes and small time delays.
3.
@@ -1971,8 +1996,8 @@ User is now able to specify Soundtrap date/time format
@@ -2126,8 +2151,8 @@ to colour them by the colour assigned to the group.
Using Old PAMGuard Configurations
-
Old PAMGuard configurations will work with the
-new version. However please note the following:
+
Old PAMGuard configurations will work with
+the new version. However please note the following:
1.
@@ -3147,8 +3172,8 @@ correctly calculated in real world coordinates.
Wild ArcGIS Interface (Maps and Mapping group)
This module, developed by Michael Oswald / SWFSC, allows
-PAMGuard to output data in a format suitable for display in the WILD ArcGIS
-marine mammal survey package. See online help for details.
+PAMGuard to output data in a format suitable for display in the WILD ArcGIS marine
+mammal survey package. See online help for details.
Alarms (Utilities Group)
@@ -3288,8 +3313,8 @@ whistle.
forms which did not display correctly in 'night' mode.
Bug fix in simulator. No longer
stops or fails to start generating sounds.
Bug in noise band monitor.
@@ -3373,9 +3398,9 @@ databases.
National Instruments cards: Added
code in support of the new x-series devices.
-
ASIO sound cards: Added support for
-a new open source ASIO driver system (jasiohost). The old system has been left
-in place for now while we assess users response to the newer system.
+
ASIO sound cards: Added support
+for a new open source ASIO driver system (jasiohost). The old system has been
+left in place for now while we assess users response to the newer system.
AIS
@@ -3450,14 +3475,14 @@ have implemented database storage for output of these modules.
'
-Radar Display: A bug which stopped the radar display from correctly
-displaying bearings to whistles from arrays containing more than two hydrophone
-elements has been fixed.
+Radar Display: A bug which stopped the radar display from correctly displaying
+bearings to whistles from arrays containing more than two hydrophone elements
+has been fixed.
'
-Database Speed: A substantial rewriting of some of the indexing methods in
-the database module has led to a significant increase in the speed at which
+Database Speed: A substantial rewriting of some of the indexing methods
+in the database module has led to a significant increase in the speed at which
data are written to the database (orders of magnitude for large databases).
This is having a significant impact on the overall reliability of the software.
Other changes have increased the speed (again by orders of magnitude) at which
@@ -3564,8 +3589,8 @@ Symbol'>''
-Bearings can be calculated using the envelope of the waveform rather
-than the full waveform. The waveform or envelope can also be filtered prior to
+Bearings can be calculated using the envelope of the waveform rather than
+the full waveform. The waveform or envelope can also be filtered prior to
bearing calculation.
''
-Wigner time-frequency plot for the click detector. Clicks are NOT
-automatically displayed in the Winger window as they are for the waveform and
-spectrum windows since the Wigner transformation is very time consuming. Clicks
-will be displayed if you select them with the mouse.
+Wigner time-frequency plot for the click detector. Clicks are NOT automatically
+displayed in the Winger window as they are for the waveform and spectrum
+windows since the Wigner transformation is very time consuming. Clicks will be
+displayed if you select them with the mouse.
'
@@ -4392,9 +4417,9 @@ a serial port
New Likelihood detector.
-
The likelihood detector module is an implementation of a likelihood
-ratio test with flexible algorithms and configuration to estimate likelihood.
-It is
+
The likelihood detector module is an implementation of a
+likelihood ratio test with flexible algorithms and configuration to estimate
+likelihood. It is
suitable for detecting both short duration Odontocete clicks
(Sperm, Beaked, etc.) as well as moderate duration Mysticete calls (Humpback,
@@ -4676,8 +4701,7 @@ Symbol'>''1.0Beta 22 Jan 2008 -
-Pamguard starts two releases, core and beta release,
+class=Heading2Char>1.0Beta 22 Jan 2008 - Pamguard
+starts two releases, core and beta release,
this is the beta release
@@ -4739,8 +4763,8 @@ Symbol'>''
-Multi animal click tracking with least squares fit calculation of
-position on map
+Multi animal click tracking with least squares fit calculation of position
+on map
'
diff --git a/dependency-reduced-pom.xml b/dependency-reduced-pom.xml
index a7c5eb80..a2dc25e4 100644
--- a/dependency-reduced-pom.xml
+++ b/dependency-reduced-pom.xml
@@ -4,7 +4,7 @@
org.pamguardPamguardPamguard Java12+
- 2.02.07b
+ 2.02.07abPamguard for Java 12+, using Maven to control dependcieswww.pamguard.org
diff --git a/pom.xml b/pom.xml
index ea1d1978..d91d13ce 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1,12 +1,12 @@
-
- 4.0.0
- org.pamguard
- Pamguard
- 2.02.07b
- Pamguard Java12+
- Pamguard for Java 12+, using Maven to control dependcies
+ 4.0.0
+ org.pamguard
+ Pamguard
+ 2.02.07ab
+ Pamguard Java12+
+ Pamguard for Java 12+, using Maven to control dependcieswww.pamguard.orgSea Mammal Research Unit, University of St. Andrews
@@ -14,240 +14,227 @@
-
- 16
- 11
- 11
-
+
+ 16
+ 11
+ 11
+
-
- src
- ${project.basedir}/src/test/java
+
+ src
+
+
+ src
+
+ **/*.java
+ jars/*.*
+
+
+
+
+
+
+
-
-
- src
-
- **/*.java
- jars/*.*
-
-
-
+
+ org.apache.maven.plugins
+ maven-compiler-plugin
+ 3.8.1
+
-
+
+ 11
+ jdt
+
+ .settings/org.eclipse.jdt.core.prefs
+
+
-
-
+
+
+
+ org.eclipse.tycho
+ tycho-compiler-jdt
+ 1.5.1
+
+
+
+
+
+
+ org.openjfx
+ javafx-maven-plugin
+ 0.0.6
+
+ 17
+ 17
+ 17
+
+
+
+
+
+
+ org.apache.maven.plugins
+ maven-shade-plugin
+ 3.2.1
+
+
+
+
+
+
+
+ package
+
+ shade
+
+
+
+
+
+ pamguard.Pamguard
+ .
+ Resources/pgBlankSplash.png
+
+
+
+
+
+
+ *:*
+
+ META-INF/*.SF
+ META-INF/*.DSA
+ META-INF/*.RSA
+
+
+
+
+
+
+
-
+
+
+ com.github.marschall
+ jdeps-maven-plugin
+ 0.5.1
+
-
- org.apache.maven.plugins
- maven-compiler-plugin
- 3.11.0
-
-
-
- 11
- jdt
-
- .settings/org.eclipse.jdt.core.prefs
-
-
-
-
-
-
- org.eclipse.tycho
- tycho-compiler-jdt
- 1.5.1
-
-
-
-
-
-
- org.openjfx
- javafx-maven-plugin
- 0.0.6
-
- 17
- 17
- 17
-
-
-
-
-
-
- org.apache.maven.plugins
- maven-shade-plugin
- 3.4.1
-
-
-
-
-
-
-
- package
-
- shade
-
-
-
-
-
- pamguard.Pamguard
- .
- Resources/pgBlankSplash.png
-
-
-
-
-
-
- *:*
-
- META-INF/*.SF
- META-INF/*.DSA
- META-INF/*.RSA
-
-
-
-
-
-
-
-
-
-
- com.github.marschall
- jdeps-maven-plugin
- 0.5.1
-
-
-
-
- org.apache.maven.plugins
- maven-dependency-plugin
- 3.5.0
-
-
- copy-dependencies
- package
-
- copy-dependencies
-
-
- ${project.build.directory}/tempDependencies
- false
- false
- true
-
-
-
-
-
-
-
-
-
-
-
-
- org.apache.maven.plugins
- maven-surefire-plugin
- 2.22.0
-
-
- org.apache.maven.plugins
- maven-failsafe-plugin
- 2.22.0
-
-
-
-
-
-
-
-
- org.eclipse.m2e
- lifecycle-mapping
- 1.0.0
-
-
-
-
-
-
- org.apache.maven.plugins
-
-
- maven-dependency-plugin
-
-
- [3.1.1,)
-
-
-
- purge-local-repository
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- com.github.marschall
- jdeps-maven-plugin
- 0.5.1
-
-
-
-
-
-
-
-
-
+
+
+ org.apache.maven.plugins
+ maven-dependency-plugin
+ 3.1.1
+
+
+ copy-dependencies
+ package
+
+ copy-dependencies
+
+
+ ${project.build.directory}/tempDependencies
+ false
+ false
+ true
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ org.eclipse.m2e
+ lifecycle-mapping
+ 1.0.0
+
+
+
+
+
+
+ org.apache.maven.plugins
+
+
+ maven-dependency-plugin
+
+
+ [3.1.1,)
+
+
+
+ purge-local-repository
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ com.github.marschall
+ jdeps-maven-plugin
+ 0.5.1
+
+
+
+
+
+
+
+
+
repo
@@ -259,568 +246,603 @@
file://${project.basedir}/repo
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- false
-
- unidata-all
- Unidata netCDF
- https://artifacts.unidata.ucar.edu/repository/unidata-all/
-
-
-
-
-
-
-
- false
-
- bedatadriven
- bedatadriven_renjin
- https://nexus.bedatadriven.com/content/groups/public/
-
-
-
-
- false
-
- talan
- talan
- https://nexus.talanlabs.com/content/repositories/releases/
-
-
-
- central
- https://repo1.maven.org/maven2
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ false
+
+ unidata-all
+ Unidata netCDF
+ https://artifacts.unidata.ucar.edu/repository/unidata-all/
+
+
+
+
+
+
+
+ false
+
+ bedatadriven
+ bedatadriven_renjin
+ https://nexus.bedatadriven.com/content/groups/public/
+
+
+
+
+ false
+
+ talan
+ talan
+ https://nexus.talanlabs.com/content/repositories/releases/
+
+
+
+ central
+ https://repo1.maven.org/maven2
+
+
-
-
+
+
-
-
-
- org.junit.jupiter
- junit-jupiter-engine
- 5.9.2
-
+
+
+ io.github.macster110
+ jpamutils
+ 0.0.56
+
+
+
+
+ io.github.macster110
+ jdl4pam
+ 0.0.94
+
+
+
+
+ gov.nist.math
+ jama
+ 1.0.3
+
+
+
+
+
+ org.openjfx
+ javafx-controls
+ ${javafx.version}
+
+
+
+
+
+ org.openjfx
+ javafx-swing
+ ${javafx.version}
+
-
- org.junit.platform
- junit-platform-runner
- 1.9.2
- test
-
+
+
+
+ org.openjfx
+ javafx-media
+ ${javafx.version}
+
+
+
+
+ org.openjfx
+ javafx-web
+ ${javafx.version}
+
+
+
+
+ net.synedra
+ validatorfx
+ 0.4.0
+
+
+
+
+ org.apache.commons
+ commons-compress
+ 1.19
+
-
-
- io.github.macster110
- jpamutils
- 0.0.56
-
-
-
-
- io.github.macster110
- jdl4pam
- 0.0.96
-
-
-
-
- gov.nist.math
- jama
- 1.0.3
-
-
-
-
-
- org.openjfx
- javafx-controls
- ${javafx.version}
-
-
-
-
-
- org.openjfx
- javafx-swing
- ${javafx.version}
-
-
-
-
-
- org.openjfx
- javafx-media
- ${javafx.version}
-
-
-
-
-
- org.openjfx
- javafx-web
- ${javafx.version}
-
-
-
-
- net.synedra
- validatorfx
- 0.4.0
-
-
-
-
- org.apache.commons
- commons-compress
- 1.19
-
-
-
-
- org.apache.commons
- commons-csv
- 1.7
-
-
-
-
- commons-io
- commons-io
- 2.6
-
-
-
-
- org.apache.commons
- commons-lang3
- 3.9
-
-
-
-
- org.apache.commons
- commons-math3
- 3.6.1
-
-
-
-
- org.apache.commons
- commons-math
- 2.2
-
-
-
-
- commons-net
- commons-net
- 3.6
-
-
-
-
- org.controlsfx
- controlsfx
- 11.0.0
-
-
- org.openjfx
- javafx-base
-
-
- org.openjfx
- javafx-controls
-
-
- org.openjfx
- javafx-graphics
-
-
- org.openjfx
- javafx-media
-
-
- org.openjfx
- javafx-web
-
-
-
-
-
-
-
- org.kordamp.ikonli
- ikonli-javafx
- 12.2.0
-
-
-
-
-
-
-
- org.kordamp.ikonli
- ikonli-materialdesign2-pack
- 12.2.0
-
-
-
-
- net.sf.geographiclib
- GeographicLib-Java
- 1.50
-
-
-
-
- org.jogamp.gluegen
- gluegen-rt-main
- 2.3.2
-
-
-
-
-
-
-
-
- com.healthmarketscience.jackcess
- jackcess
- 3.0.1
-
-
-
-
- com.fasterxml.jackson.core
- jackson-databind
- 2.10.1
-
-
-
-
- org.jogamp.jogl
- jogl-all-main
- 2.3.2
-
-
-
-
-
-
- org.jflac
- jflac-codec
- 1.5.2
-
-
-
-
- javax.help
- javahelp
- 2.0.05
-
-
-
-
- net.java.dev.jna
- jna
- 5.5.0
-
+
+
+ org.apache.commons
+ commons-csv
+ 1.7
+
+
+
+
+ commons-io
+ commons-io
+ 2.6
+
+
+
+
+ org.apache.commons
+ commons-lang3
+ 3.9
+
+
+
+
+ org.apache.commons
+ commons-math3
+ 3.6.1
+
+
+
+
+ org.apache.commons
+ commons-math
+ 2.2
+
+
+
+
+ commons-net
+ commons-net
+ 3.6
+
+
+
+
+ org.controlsfx
+ controlsfx
+ 11.0.0
+
+
+ org.openjfx
+ javafx-base
+
+
+ org.openjfx
+ javafx-controls
+
+
+ org.openjfx
+ javafx-graphics
+
+
+ org.openjfx
+ javafx-media
+
+
+ org.openjfx
+ javafx-web
+
+
+
+
+
+
+
+ org.kordamp.ikonli
+ ikonli-javafx
+ 12.2.0
+
+
+
+
+
+
+
+ org.kordamp.ikonli
+ ikonli-materialdesign2-pack
+ 12.2.0
+
+
+
+
+ net.sf.geographiclib
+ GeographicLib-Java
+ 1.50
+
+
+
+
+ org.jogamp.gluegen
+ gluegen-rt-main
+ 2.3.2
+
+
+
+
+
+
+
+ com.healthmarketscience.jackcess
+ jackcess
+ 3.0.1
+
+
+
+
+ com.fasterxml.jackson.core
+ jackson-databind
+ 2.10.1
+
+
+
+
+ org.jogamp.jogl
+ jogl-all-main
+ 2.3.2
+
+
+
+
+
+
+ org.jflac
+ jflac-codec
+ 1.5.2
+
+
+
+
+ javax.help
+ javahelp
+ 2.0.05
+
+
+
+
+ net.java.dev.jna
+ jna
+ 5.5.0
+
+
-
- net.java.dev.jna
- jna-platform
- 5.5.0
-
+
+ net.java.dev.jna
+ jna-platform
+ 5.5.0
+
+
+
+
+ com.jcraft
+ jsch
+ 0.1.55
+
+
+
+
+ com.fazecast
+ jSerialComm
+ 2.5.3
+
-
-
- com.jcraft
- jsch
- 0.1.55
-
+
+
+ edu.emory.mathcs
+ JTransforms
+ 2.4
+
-
-
- com.fazecast
- jSerialComm
- 2.5.3
-
+
+
+ com.sun.mail
+ javax.mail
+ 1.6.2
+
-
-
- edu.emory.mathcs
- JTransforms
- 2.4
-
+
+
+ com.diffplug.matsim
+ matfilerw
+ 3.1.1
+
-
-
- com.sun.mail
- javax.mail
- 1.6.2
-
+
+
+ com.drewnoakes
+ metadata-extractor
+ 2.12.0
+
-
-
- com.diffplug.matsim
- matfilerw
- 3.1.1
-
+
+
+ mysql
+ mysql-connector-java
+ 8.0.18
+
-
-
- com.drewnoakes
- metadata-extractor
- 2.12.0
-
+
+
+
+
+ com.google.protobuf
+ protobuf-java
+ 3.17.0
+
-
-
- mysql
- mysql-connector-java
- 8.0.18
-
-
-
-
-
-
+
+
+
+ edu.ucar
+ netcdfAll
+ 4.6.14
+
+ com.google.protobufprotobuf-java
- 3.17.0
-
+
+
+ com.google.protobuf
+ protobuf-java-util
+
+
+ compile
+
-
-
-
- edu.ucar
- netcdfAll
- 4.6.14
-
-
- com.google.protobuf
- protobuf-java
-
-
- com.google.protobuf
- protobuf-java-util
-
-
- compile
-
+
+
+ com.opencsv
+ opencsv
+ 5.0
+
-
-
- com.opencsv
- opencsv
- 5.0
-
+
-
+
+
+ org.postgresql
+ postgresql
+ 42.2.24
+
-
-
- org.postgresql
- postgresql
- 42.2.24
-
+
+
+ org.renjin
+ renjin-script-engine
+ 0.9.2725
+
+
+
+
+ org.slf4j
+ slf4j-api
+ 1.8.0-beta4
+
+
+
+
+
+
+ org.slf4j
+ slf4j-nop
+ 1.8.0-beta4
+
+
+
-
-
- org.renjin
- renjin-script-engine
- 0.9.2725
-
+
+
+
+
+ org.docx4j
+ docx4j-JAXB-ReferenceImpl
+ 11.1.3
+
+
-
-
- org.slf4j
- slf4j-api
- 1.8.0-beta4
-
+
+
+ org.xerial
+ sqlite-jdbc
+ 3.28.0
+
+
+
+ net.sf.ucanaccess
+ ucanaccess
+ 4.0.4
+
+
+
+ nz.ac.waikato.cms.weka
+ weka-dev
+ 3.7.7
+
-
-
- org.slf4j
- slf4j-nop
- 1.8.0-beta4
-
+
+
+ javax.vecmath
+ vecmath
+ 1.5.2
+
+
+
+
-
-
-
-
-
- org.docx4j
- docx4j-JAXB-ReferenceImpl
- 11.1.3
-
-
-
-
-
- org.xerial
- sqlite-jdbc
- 3.28.0
-
-
-
-
- net.sf.ucanaccess
- ucanaccess
- 4.0.4
-
-
-
-
- nz.ac.waikato.cms.weka
- weka-dev
- 3.7.7
-
-
-
-
- javax.vecmath
- vecmath
- 1.5.2
-
-
-
-
-
-
-
-
- pamguard.org
- x3
- 2.2.0
-
-
+
+
+ pamguard.org
+ x3
+ 2.2.2
+
+
-
- it.sauronsoftware
- jave
- 1.0.2
-
+
+ it.sauronsoftware
+ jave
+ 1.0.2
+
+
+
+
+ com.synthbot
+ jasiohost
+ 1.0.0
+
-
-
- com.synthbot
- jasiohost
- 1.0.0
-
+
+
+
+
+
+ org.springframework
+ spring-core
+ 5.2.3.RELEASE
+
+
+
+
+
+ com.1stleg
+ jnativehook
+ 2.1.0
+
+
+
+
+
+ org.swinglabs.swingx
+ swingx-all
+ 1.6.5-1
+
-
-
-
-
-
- org.springframework
- spring-core
- 5.2.3.RELEASE
-
-
-
-
-
- com.1stleg
- jnativehook
- 2.1.0
-
-
-
-
-
- org.swinglabs.swingx
- swingx-all
- 1.6.5-1
-
-
-
-
- io.github.mkpaz
- atlantafx-base
- 1.0.0
-
-
-
-
+
+
+ io.github.mkpaz
+ atlantafx-base
+ 1.0.0
+
+
+
+
\ No newline at end of file
diff --git a/repo/pamguard/org/x3/2.2.1/_remote.repositories b/repo/pamguard/org/x3/2.2.1/_remote.repositories
new file mode 100644
index 00000000..51f219b8
--- /dev/null
+++ b/repo/pamguard/org/x3/2.2.1/_remote.repositories
@@ -0,0 +1,4 @@
+#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
+#Mon Apr 24 10:32:52 BST 2023
+x3-2.2.1.jar>=
+x3-2.2.1.pom>=
diff --git a/repo/pamguard/org/x3/2.2.1/x3-2.2.1.jar b/repo/pamguard/org/x3/2.2.1/x3-2.2.1.jar
new file mode 100644
index 00000000..a80ce5cd
Binary files /dev/null and b/repo/pamguard/org/x3/2.2.1/x3-2.2.1.jar differ
diff --git a/repo/pamguard/org/x3/2.2.1/x3-2.2.1.pom b/repo/pamguard/org/x3/2.2.1/x3-2.2.1.pom
new file mode 100644
index 00000000..ff69a57c
--- /dev/null
+++ b/repo/pamguard/org/x3/2.2.1/x3-2.2.1.pom
@@ -0,0 +1,9 @@
+
+
+ 4.0.0
+ pamguard.org
+ x3
+ 2.2.1
+ POM was created from install:install-file
+
diff --git a/repo/pamguard/org/x3/2.2.2/_remote.repositories b/repo/pamguard/org/x3/2.2.2/_remote.repositories
new file mode 100644
index 00000000..34e5eed9
--- /dev/null
+++ b/repo/pamguard/org/x3/2.2.2/_remote.repositories
@@ -0,0 +1,4 @@
+#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
+#Mon May 08 15:05:07 BST 2023
+x3-2.2.2.jar>=
+x3-2.2.2.pom>=
diff --git a/repo/pamguard/org/x3/2.2.2/m2e-lastUpdated.properties b/repo/pamguard/org/x3/2.2.2/m2e-lastUpdated.properties
new file mode 100644
index 00000000..1a03a905
--- /dev/null
+++ b/repo/pamguard/org/x3/2.2.2/m2e-lastUpdated.properties
@@ -0,0 +1,15 @@
+#Sat May 06 19:10:45 BST 2023
+bedatadriven|https\://nexus.bedatadriven.com/content/groups/public/|javadoc=1683396645122
+repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo|javadoc=1683137958887
+repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo|javadoc=1683137015763
+talan|https\://nexus.talanlabs.com/content/repositories/releases/|sources=1683368185314
+bedatadriven|https\://nexus.bedatadriven.com/content/groups/public/|sources=1683368185314
+unidata-all|https\://artifacts.unidata.ucar.edu/repository/unidata-all/|sources=1683368185314
+repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardTethys/repo|sources=1683368185314
+repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo|sources=1683136978098
+repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo|sources=1683137602295
+talan|https\://nexus.talanlabs.com/content/repositories/releases/|javadoc=1683396645122
+central|https\://repo1.maven.org/maven2|sources=1683368185314
+unidata-all|https\://artifacts.unidata.ucar.edu/repository/unidata-all/|javadoc=1683396645122
+repo|file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardTethys/repo|javadoc=1683396645122
+central|https\://repo1.maven.org/maven2|javadoc=1683396645122
diff --git a/repo/pamguard/org/x3/2.2.2/x3-2.2.2-javadoc.jar.lastUpdated b/repo/pamguard/org/x3/2.2.2/x3-2.2.2-javadoc.jar.lastUpdated
new file mode 100644
index 00000000..77a68e30
--- /dev/null
+++ b/repo/pamguard/org/x3/2.2.2/x3-2.2.2-javadoc.jar.lastUpdated
@@ -0,0 +1,16 @@
+#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
+#Sat May 06 19:10:45 BST 2023
+@default-talan-https\://nexus.talanlabs.com/content/repositories/releases/.lastUpdated=1683137015553
+file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo/.error=
+https\://repo1.maven.org/maven2/.error=
+file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo/.lastUpdated=1683137958884
+file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardTethys/repo/.lastUpdated=1683396645119
+https\://artifacts.unidata.ucar.edu/repository/unidata-all/.error=
+https\://nexus.bedatadriven.com/content/groups/public/.error=
+file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardTethys/repo/.error=
+https\://artifacts.unidata.ucar.edu/repository/unidata-all/.lastUpdated=1683137015146
+https\://repo1.maven.org/maven2/.lastUpdated=1683137015762
+https\://nexus.talanlabs.com/content/repositories/releases/.error=Could not transfer artifact pamguard.org\:x3\:jar\:javadoc\:2.2.2 from/to talan (https\://nexus.talanlabs.com/content/repositories/releases/)\: Access denied to https\://nexus.talanlabs.com/content/repositories/releases/pamguard/org/x3/2.2.2/x3-2.2.2-javadoc.jar. Error code 401, Unauthorized
+file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo/.lastUpdated=1683137014154
+file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo/.error=
+https\://nexus.bedatadriven.com/content/groups/public/.lastUpdated=1683137015343
diff --git a/repo/pamguard/org/x3/2.2.2/x3-2.2.2-sources.jar.lastUpdated b/repo/pamguard/org/x3/2.2.2/x3-2.2.2-sources.jar.lastUpdated
new file mode 100644
index 00000000..df14958c
--- /dev/null
+++ b/repo/pamguard/org/x3/2.2.2/x3-2.2.2-sources.jar.lastUpdated
@@ -0,0 +1,16 @@
+#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
+#Sat May 06 11:16:25 BST 2023
+@default-talan-https\://nexus.talanlabs.com/content/repositories/releases/.lastUpdated=1683136977699
+file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo/.error=
+https\://repo1.maven.org/maven2/.error=
+file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo/.lastUpdated=1683137602291
+file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardTethys/repo/.lastUpdated=1683368185312
+https\://artifacts.unidata.ucar.edu/repository/unidata-all/.error=
+https\://nexus.bedatadriven.com/content/groups/public/.error=
+file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardTethys/repo/.error=
+https\://artifacts.unidata.ucar.edu/repository/unidata-all/.lastUpdated=1683136976818
+https\://repo1.maven.org/maven2/.lastUpdated=1683136978096
+https\://nexus.talanlabs.com/content/repositories/releases/.error=Could not transfer artifact pamguard.org\:x3\:jar\:sources\:2.2.2 from/to talan (https\://nexus.talanlabs.com/content/repositories/releases/)\: Access denied to https\://nexus.talanlabs.com/content/repositories/releases/pamguard/org/x3/2.2.2/x3-2.2.2-sources.jar. Error code 401, Unauthorized
+file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo/.lastUpdated=1683136975590
+file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardPAMGuard/repo/.error=
+https\://nexus.bedatadriven.com/content/groups/public/.lastUpdated=1683136977227
diff --git a/repo/pamguard/org/x3/2.2.2/x3-2.2.2.jar b/repo/pamguard/org/x3/2.2.2/x3-2.2.2.jar
new file mode 100644
index 00000000..63f5ecb2
Binary files /dev/null and b/repo/pamguard/org/x3/2.2.2/x3-2.2.2.jar differ
diff --git a/repo/pamguard/org/x3/2.2.2/x3-2.2.2.pom b/repo/pamguard/org/x3/2.2.2/x3-2.2.2.pom
new file mode 100644
index 00000000..88abe06b
--- /dev/null
+++ b/repo/pamguard/org/x3/2.2.2/x3-2.2.2.pom
@@ -0,0 +1,9 @@
+
+
+ 4.0.0
+ pamguard.org
+ x3
+ 2.2.2
+ POM was created from install:install-file
+
diff --git a/repo/pamguard/org/x3/2.2.2/x3-2.2.2.pom.lastUpdated b/repo/pamguard/org/x3/2.2.2/x3-2.2.2.pom.lastUpdated
new file mode 100644
index 00000000..41403db8
--- /dev/null
+++ b/repo/pamguard/org/x3/2.2.2/x3-2.2.2.pom.lastUpdated
@@ -0,0 +1,12 @@
+#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
+#Thu May 04 11:43:09 BST 2023
+@default-talan-https\://nexus.talanlabs.com/content/repositories/releases/.lastUpdated=1683196988926
+file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo/.error=
+https\://repo1.maven.org/maven2/.error=
+file\://C\:\\Users\\dg50\\source\\repos\\PAMGuardDG/repo/.lastUpdated=1683196987083
+https\://artifacts.unidata.ucar.edu/repository/unidata-all/.error=
+https\://nexus.bedatadriven.com/content/groups/public/.error=
+https\://nexus.bedatadriven.com/content/groups/public/.lastUpdated=1683196988663
+https\://artifacts.unidata.ucar.edu/repository/unidata-all/.lastUpdated=1683196988452
+https\://repo1.maven.org/maven2/.lastUpdated=1683196989198
+https\://nexus.talanlabs.com/content/repositories/releases/.error=Could not transfer artifact pamguard.org\:x3\:pom\:2.2.2 from/to talan (https\://nexus.talanlabs.com/content/repositories/releases/)\: Access denied to https\://nexus.talanlabs.com/content/repositories/releases/pamguard/org/x3/2.2.2/x3-2.2.2.pom. Error code 401, Unauthorized
diff --git a/src/Acquisition/AcquisitionControl.java b/src/Acquisition/AcquisitionControl.java
index 150fc411..406796c8 100644
--- a/src/Acquisition/AcquisitionControl.java
+++ b/src/Acquisition/AcquisitionControl.java
@@ -62,6 +62,8 @@ import Acquisition.sud.SUDNotificationManager;
import Array.ArrayManager;
import Array.PamArray;
import Array.Preamplifier;
+import PamController.DataInputStore;
+import PamController.InputStoreInfo;
import PamController.OfflineFileDataStore;
import PamController.PamControlledUnit;
import PamController.PamControlledUnitGUI;
@@ -71,6 +73,8 @@ import PamController.PamControllerInterface;
import PamController.PamGUIManager;
import PamController.PamSettingManager;
import PamController.PamSettings;
+import PamController.RawInputControlledUnit;
+import PamController.fileprocessing.StoreStatus;
import PamModel.PamModel;
import PamModel.SMRUEnable;
import PamUtils.FrequencyFormat;
@@ -100,7 +104,7 @@ import PamguardMVC.dataOffline.OfflineDataLoadInfo;
* @see Acquisition.DaqSystem
*
*/
-public class AcquisitionControl extends PamControlledUnit implements PamSettings, OfflineFileDataStore {
+public class AcquisitionControl extends RawInputControlledUnit implements PamSettings, OfflineFileDataStore, DataInputStore {
protected ArrayList systemList;
@@ -849,4 +853,39 @@ public class AcquisitionControl extends PamControlledUnit implements PamSettings
return sudNotificationManager;
}
+
+ @Override
+ public int getRawInputType() {
+ DaqSystem system = acquisitionProcess.getRunningSystem();
+ if (system == null) {
+ return RAW_INPUT_UNKNOWN;
+ }
+ else {
+ return system.isRealTime() ? RAW_INPUT_REALTIME : RAW_INPUT_FILEARCHIVE;
+ }
+ }
+ @Override
+ public InputStoreInfo getStoreInfo(boolean detail) {
+ return getDaqProcess().getStoreInfo(detail);
+ }
+ @Override
+ public boolean setAnalysisStartTime(long startTime) {
+ return getDaqProcess().setAnalysisStartTime(startTime);
+ }
+
+ /**
+ * Very specific command handler for batch status which will only work
+ * with the folderinputSystem.
+ * @return
+ */
+ public String getBatchStatus() {
+ DaqSystem runningSystem = getAcquisitionProcess().getRunningSystem();
+ if (runningSystem instanceof FolderInputSystem) {
+ return ((FolderInputSystem) runningSystem).getBatchStatus();
+ }
+ else {
+ return null;
+ }
+ }
+
}
diff --git a/src/Acquisition/AcquisitionProcess.java b/src/Acquisition/AcquisitionProcess.java
index ef896874..c5e331fc 100644
--- a/src/Acquisition/AcquisitionProcess.java
+++ b/src/Acquisition/AcquisitionProcess.java
@@ -1,7 +1,9 @@
package Acquisition;
+import java.awt.Window;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
+import java.util.ArrayList;
import java.util.Collections;
import java.util.LinkedList;
import java.util.List;
@@ -19,7 +21,13 @@ import Filters.FilterBand;
import Filters.FilterParams;
import Filters.FilterType;
import Filters.IirfFilter;
+import PamController.DataInputStore;
+import PamController.InputStoreInfo;
+import PamController.OfflineDataStore;
+import PamController.PamControlledUnit;
import PamController.PamController;
+import PamController.fileprocessing.ReprocessManager;
+import PamController.fileprocessing.StoreStatus;
import PamController.status.BaseProcessCheck;
import PamController.status.ProcessCheck;
import PamDetection.RawDataUnit;
@@ -36,7 +44,10 @@ import PamguardMVC.PamProcess;
import PamguardMVC.PamRawDataBlock;
import PamguardMVC.RequestCancellationObject;
import PamguardMVC.dataOffline.OfflineDataLoadInfo;
+import dataGram.DatagramManager;
+import dataMap.OfflineDataMapPoint;
import pamScrollSystem.AbstractScrollManager;
+import pamScrollSystem.ViewLoadObserver;
/**
* Data acquisition process for all types of input device.
@@ -54,7 +65,7 @@ import pamScrollSystem.AbstractScrollManager;
* @see PamguardMVC.PamDataUnit
*
*/
-public class AcquisitionProcess extends PamProcess {
+public class AcquisitionProcess extends PamProcess implements DataInputStore {
public static final int LASTDATA = 2; // don't use zero since need to see if no notification has been received.
@@ -523,12 +534,12 @@ public class AcquisitionProcess extends PamProcess {
System.out.printf("Unable to find daq system %s\n", acquisitionControl.acquisitionParameters.daqSystemType);
return;
}
-
-
+
systemPrepared = runningSystem.prepareSystem(acquisitionControl);
}
+
@Override
public void setSampleRate(float sampleRate, boolean notify) {
acquisitionControl.acquisitionParameters.sampleRate = sampleRate;
@@ -1223,6 +1234,28 @@ public class AcquisitionProcess extends PamProcess {
public PamDataBlock getDaqStatusDataBlock() {
return daqStatusDataBlock;
}
+
+ @Override
+ public InputStoreInfo getStoreInfo(boolean detail) {
+ if (runningSystem instanceof DataInputStore) {
+ return ((DataInputStore) runningSystem).getStoreInfo(detail);
+ }
+ else {
+ return null;
+ }
+ }
+
+ @Override
+ public boolean setAnalysisStartTime(long startTime) {
+ if (runningSystem instanceof DataInputStore) {
+ return ((DataInputStore) runningSystem).setAnalysisStartTime(startTime);
+ }
+ else {
+ return false;
+ }
+ }
+
+
}
diff --git a/src/Acquisition/FileInputSystem.java b/src/Acquisition/FileInputSystem.java
index 1b15482f..2727ca05 100644
--- a/src/Acquisition/FileInputSystem.java
+++ b/src/Acquisition/FileInputSystem.java
@@ -558,7 +558,10 @@ public class FileInputSystem extends DaqSystem implements ActionListener, PamSe
// if (fileInputParameters.recentFiles == null) return false;
// if (fileInputParameters.recentFiles.size() < 1) return false;
// String fileName = fileInputParameters.recentFiles.get(0);
- return runFileAnalysis();
+ if (runFileAnalysis() == false) {
+ return false;
+ }
+ return true;
}
public File getCurrentFile() {
@@ -737,6 +740,8 @@ public class FileInputSystem extends DaqSystem implements ActionListener, PamSe
@Override
public boolean startSystem(AcquisitionControl daqControl) {
+
+
if (audioStream == null) return false;
dontStop = true;
@@ -1146,6 +1151,7 @@ public class FileInputSystem extends DaqSystem implements ActionListener, PamSe
protected void fileListComplete() {
if (GlobalArguments.getParam(PamController.AUTOEXIT) != null) {
System.out.println("All sound files processed, PAMGuard can close on " + PamController.AUTOEXIT);
+ PamController.getInstance().setPamStatus(PamController.PAM_COMPLETE);
PamController.getInstance().batchProcessingComplete();
}
}
diff --git a/src/Acquisition/FolderInputSystem.java b/src/Acquisition/FolderInputSystem.java
index 92ee94d7..3eed96ec 100644
--- a/src/Acquisition/FolderInputSystem.java
+++ b/src/Acquisition/FolderInputSystem.java
@@ -33,6 +33,8 @@ import pamguard.GlobalArguments;
import Acquisition.pamAudio.PamAudioFileManager;
import Acquisition.pamAudio.PamAudioFileFilter;
import Acquisition.pamAudio.PamAudioSystem;
+import PamController.DataInputStore;
+import PamController.InputStoreInfo;
import PamController.PamControlledUnitSettings;
import PamController.PamController;
import PamController.PamSettings;
@@ -48,6 +50,7 @@ import PamView.dialog.PamGridBagContraints;
import PamView.dialog.PamLabel;
import PamView.panel.PamPanel;
import PamView.panel.PamProgressBar;
+import PamguardMVC.debug.Debug;
/**
* Read multiple files in sequence. Options exist to either pause and
@@ -57,7 +60,7 @@ import PamView.panel.PamProgressBar;
* @author Doug Gillespie
*
*/
-public class FolderInputSystem extends FileInputSystem implements PamSettings{
+public class FolderInputSystem extends FileInputSystem implements PamSettings, DataInputStore {
// Timer timer;
public static final String daqType = "File Folder Acquisition System";
@@ -116,7 +119,7 @@ public class FolderInputSystem extends FileInputSystem implements PamSettings{
if (folderInputParameters == null)
setFolderInputParameters(new FolderInputParameters(getSystemType()));
// PamSettingManager.getInstance().registerSettings(this); //calling super already registers this in the FileInputSystem constructor
- checkComandLine();
+// checkComandLine();
makeSelFileList();
newFileTimer = new Timer(1000, new RestartTimer());
newFileTimer.setRepeats(false);
@@ -126,18 +129,24 @@ public class FolderInputSystem extends FileInputSystem implements PamSettings{
/**
* Check to see if acquisition source folder was set in the command line.
*/
- private void checkComandLine() {
+ private String[] checkComandLineFolder() {
String globalFolder = GlobalArguments.getParam(GlobalWavFolderArg);
+ Debug.out.println("Checking -wavfilefolder option: is " + globalFolder);
if (globalFolder == null) {
- return;
+ return null;
}
// see if it at least exists, though will we want to do this for Network folders ?
File aFile = new File(globalFolder);
if (aFile.exists() == false) {
- System.err.println("Command line folder does not exist: " + globalFolder);
+ System.err.printf("Command line wav folder \"%s\" does not exist", globalFolder);
+// return null;
}
String[] selList = {globalFolder};
- folderInputParameters.setSelectedFiles(selList);
+// folderInputParameters.setSelectedFiles(selList);
+ // need to immediately make the allfiles list since it's about to get used by the reprocess manager
+ // need to worry about how to wait for this since it's starting in a different thread.
+ //makeSelFileList();
+ return selList;
}
/**
@@ -290,15 +299,24 @@ public class FolderInputSystem extends FileInputSystem implements PamSettings{
}
/**
- * Make a list of wav files within a folder.
+ * Make a list of wav files within a folder. In some circumstances this can be a list
+ * of actual files in a folder. Also needs to handle the possibility of it using
+ * a globally set folder name.
* @return flag to indicate...nothing?
*/
public int makeSelFileList() {
- if (fileInputParameters.recentFiles == null || fileInputParameters.recentFiles.size() < 1) {
- return 0;
+ String[] selection = checkComandLineFolder();
+
+ if (selection == null) {
+ if (fileInputParameters.recentFiles == null || fileInputParameters.recentFiles.size() < 1) {
+ return 0;
+ }
+ selection = folderInputParameters.getSelectedFiles();
+ }
+ if (selection.length > 0) {
+ System.out.println("FolderInputSystem.makeSelFileList(): Searching for sound files in " + selection[0]);
}
- String[] selection = folderInputParameters.getSelectedFiles();
return makeSelFileList(selection);
}
@@ -827,5 +845,93 @@ public class FolderInputSystem extends FileInputSystem implements PamSettings{
folderInputPane.setParams(folderInputParameters);
}
+ @Override
+ public InputStoreInfo getStoreInfo(boolean detail) {
+ if (allFiles == null || allFiles.size() == 0) {
+ return null;
+ }
+ WavFileType firstFile = allFiles.get(0);
+ long firstFileStart = getFileStartTime(firstFile.getAbsoluteFile());
+ WavFileType lastFile = allFiles.get(allFiles.size()-1);
+ long lastFileStart = getFileStartTime(lastFile.getAbsoluteFile());
+ lastFile.getAudioInfo();
+ long lastFileEnd = (long) (lastFileStart + lastFile.getDurationInSeconds()*1000.);
+ InputStoreInfo storeInfo = new InputStoreInfo(acquisitionControl, allFiles.size(), firstFileStart, lastFileStart, lastFileEnd);
+ if (detail) {
+ long[] allFileStarts = new long[allFiles.size()];
+ for (int i = 0; i < allFiles.size(); i++) {
+ allFileStarts[i] = getFileStartTime(allFiles.get(i).getAbsoluteFile());
+ if (allFileStarts[i] < firstFileStart) {
+// System.out.printf("Swap first file from %s to %s\n", firstFile.getName(), allFiles.get(i).getName());
+ firstFile = allFiles.get(i);
+ firstFileStart = allFileStarts[i];
+ }
+ if (allFileStarts[i] > lastFileEnd) {
+// System.out.printf("Swap last file from %s to %s\n", lastFile.getName(), allFiles.get(i).getName());
+ lastFile = allFiles.get(i);
+ lastFileEnd = allFileStarts[i] + (long) (lastFile.getDurationInSeconds()*1000.);
+ }
+ }
+ storeInfo.setFirstFileStart(firstFileStart); // just incase changed.
+ storeInfo.setLastFileEnd(lastFileEnd); // just incase changed
+ storeInfo.setFileStartTimes(allFileStarts);
+ }
+ return storeInfo;
+ }
+
+ @Override
+ public boolean setAnalysisStartTime(long startTime) {
+ /**
+ * Called from the reprocess manager just before PAMGuard starts with a time
+ * we want to process from. This should be equal to the start of one of the files
+ * so all we have to do (in principle) is to set the currentfile to that index and
+ * processing will continue from there.
+ */
+ if (allFiles == null || allFiles.size() == 0) {
+ System.out.println("Daq setanal start time: no files to check against");
+ return false;
+ }
+ System.out.printf("setAnalysisStarttTime: checking %d files for start time of %s\n", allFiles.size(), PamCalendar.formatDBDateTime(startTime));
+ /*
+ * If the starttime is maxint then there is nothing to do, but we do need to set the file index
+ * correctly to not over confuse the batch processing system.
+ */
+ long lastFileTime = getFileStartTime(allFiles.get(allFiles.size()-1).getAbsoluteFile());
+ if (startTime > lastFileTime) {
+ currentFile = allFiles.size();
+ System.out.println("Folder Acquisition processing is complete and no files require processing");
+ return true;
+ }
+ for (int i = 0; i < allFiles.size(); i++) {
+ long fileStart = getFileStartTime(allFiles.get(i).getAbsoluteFile());
+ if (fileStart >= startTime) {
+ currentFile = i;
+ PamCalendar.setSoundFile(true);
+ if (startTime > 0) {
+ PamCalendar.setSessionStartTime(startTime);
+ System.out.printf("Sound Acquisition start processing at file %s time %s\n", allFiles.get(i).getName(),
+ PamCalendar.formatDBDateTime(fileStart));
+ }
+ return true;
+ }
+ }
+
+ return false;
+ }
+
+ /**
+ * Get a status update for batch processing.
+ */
+ public String getBatchStatus() {
+ int nFiles = 0;
+ if (allFiles != null) {
+ nFiles = allFiles.size();
+ }
+ int generalStatus = PamController.getInstance().getPamStatus();
+ File currFile = getCurrentFile();
+ String bs = String.format("%d,%d,%d,%s", nFiles,currentFile,generalStatus,currFile);
+ return bs;
+ }
+
}
diff --git a/src/Acquisition/filedate/StandardFileDate.java b/src/Acquisition/filedate/StandardFileDate.java
index aa87e973..53033f60 100644
--- a/src/Acquisition/filedate/StandardFileDate.java
+++ b/src/Acquisition/filedate/StandardFileDate.java
@@ -284,9 +284,12 @@ public class StandardFileDate implements FileDate, PamSettings {
} catch (ParseException e) {
// TODO Auto-generated catch block
// e.printStackTrace();
- System.out.println(e.getMessage());
+ System.out.println("StandardfileDate.forcedDataFormat:" + e.getMessage());
} //throws ParseException if no match
setLastFormat(forcedDateFormat);
+ if (d == null) {
+ return 0;
+ }
return d.getTime();
}
diff --git a/src/Acquisition/offlineFuncs/OfflineWavFileServer.java b/src/Acquisition/offlineFuncs/OfflineWavFileServer.java
index c1b6efa6..b10b27dc 100644
--- a/src/Acquisition/offlineFuncs/OfflineWavFileServer.java
+++ b/src/Acquisition/offlineFuncs/OfflineWavFileServer.java
@@ -17,6 +17,7 @@ import Acquisition.pamAudio.PamAudioFileManager;
import Acquisition.pamAudio.PamAudioFileLoader;
import Acquisition.pamAudio.PamAudioFileFilter;
import PamController.OfflineFileDataStore;
+import PamController.fileprocessing.StoreStatus;
import PamguardMVC.PamDataBlock;
import PamguardMVC.dataOffline.OfflineDataLoadInfo;
import dataMap.OfflineDataMap;
@@ -188,5 +189,4 @@ public class OfflineWavFileServer extends OfflineFileServer {
}
-
}
diff --git a/src/IshmaelLocator/IshLocProcess.java b/src/IshmaelLocator/IshLocProcess.java
index cdec7212..9b34462d 100644
--- a/src/IshmaelLocator/IshLocProcess.java
+++ b/src/IshmaelLocator/IshLocProcess.java
@@ -368,7 +368,7 @@ abstract public class IshLocProcess extends PamProcess implements SpectrogramMar
selectionSams = daqBlock.getSamples(startSam, (int)durationSam, channelMap);
}
catch (RawDataUnavailableException e) {
- System.out.println(e.getMessage());
+ System.out.println("RawDataUnavailableException in IshLocProcess:" + e.getMessage());
// if the raw data has already been discarded, adjust the natural lifetime to try and avoid this in the future
int newTime;
diff --git a/src/JSSHTerminal/SSHSession.java b/src/JSSHTerminal/SSHSession.java
index daf2e2f1..af5ccd56 100644
--- a/src/JSSHTerminal/SSHSession.java
+++ b/src/JSSHTerminal/SSHSession.java
@@ -287,7 +287,7 @@ public final class SSHSession implements UserInfo {
out.write(k);
out.flush();
} catch (IOException e) {
- System.out.println(e.getMessage());
+ System.out.println("IOException in SSHSession.write: " + e.getMessage());
}
}
@@ -349,7 +349,7 @@ public final class SSHSession implements UserInfo {
}
}
} catch (IOException e) {
- System.out.println(e.getMessage());
+ System.out.println("Error in SSHSession.pump: " + e.getMessage());
} finally {
closeQuietly(in);
closeQuietly(out);
diff --git a/src/NMEA/AcquireNmeaData.java b/src/NMEA/AcquireNmeaData.java
index 95a0f39b..e86d0ba2 100644
--- a/src/NMEA/AcquireNmeaData.java
+++ b/src/NMEA/AcquireNmeaData.java
@@ -203,7 +203,7 @@ public class AcquireNmeaData extends PamProcess implements ActionListener, Modul
try {
pjSerialComm = PJSerialComm.openSerialPort(params.serialPortName, params.serialPortBitsPerSecond);
} catch (PJSerialException e) {
- System.out.println(e.getMessage());
+ System.out.println("PJSerialException in AcquireNMEAData" + e.getMessage());
// WarnOnce.sho
return;
}
diff --git a/src/PamController/DataInputStore.java b/src/PamController/DataInputStore.java
new file mode 100644
index 00000000..af4f5a4b
--- /dev/null
+++ b/src/PamController/DataInputStore.java
@@ -0,0 +1,31 @@
+package PamController;
+
+/**
+ * Functions for a data input store. There is a fair bit of overlap for this and
+ * OfflineDataStore, but the OfflineDataStore is really about stuff that can provide
+ * data offline which needs mapping. This is specifically about data which will be input
+ * during 'normal operation, i.e. sound acquisition and Tritech sonar data
+ * (a plugin, but coming down the tracks at us all).
+ * @author dg50
+ * @see OfflineDataStore
+ * @See DataOutputStore
+ *
+ */
+public interface DataInputStore {
+
+ /**
+ * Get information about the input store (e.g. start times of all files).
+ * @param detail
+ * @return information about data input.
+ */
+ public InputStoreInfo getStoreInfo(boolean detail);
+
+ /**
+ * Set an analysis start time. This might get called just before
+ * processing starts, in which case
+ * @param startTime
+ * @return ok if no problems.
+ */
+ public boolean setAnalysisStartTime(long startTime);
+
+}
diff --git a/src/PamController/DataOutputStore.java b/src/PamController/DataOutputStore.java
new file mode 100644
index 00000000..a3536410
--- /dev/null
+++ b/src/PamController/DataOutputStore.java
@@ -0,0 +1,32 @@
+package PamController;
+
+import PamController.fileprocessing.StoreStatus;
+
+/**
+ * Functions for a data output store. there is a fair bit of overlap for this and
+ * OfflineDataStore, but the OfflineDataStore is really about stuff that can provide
+ * data offline which needs mapping. This is specifically about data which will be stored
+ * during 'normal operation, i.e. binary and database modules.
+ * @author dg50
+ * @see OfflineDataStore
+ * @See DataInputStore
+ *
+ */
+public interface DataOutputStore extends OfflineDataStore {
+
+ /**
+ * Get the store status, i.e. does it exist, does it contain data, if so over what date range,
+ * etc.
+ * @param getDetail
+ * @return
+ */
+ public StoreStatus getStoreStatus(boolean getDetail);
+
+ /**
+ * Delete all data from a given time, in all data streams.
+ * @param timeMillis time to delete from (anything >= this time)
+ * @return true if it seems to have worked OK. False if any errors (e.g. database or file system error).
+ */
+ public boolean deleteDataFrom(long timeMillis);
+
+}
diff --git a/src/PamController/InputStoreInfo.java b/src/PamController/InputStoreInfo.java
new file mode 100644
index 00000000..cde04c8c
--- /dev/null
+++ b/src/PamController/InputStoreInfo.java
@@ -0,0 +1,102 @@
+package PamController;
+
+import PamUtils.PamCalendar;
+
+public class InputStoreInfo {
+
+ private DataInputStore dataInputStore;
+ private int nFiles;
+ private long firstFileStart, lastFileStart, lastFileEnd;
+ private long[] fileStartTimes;
+
+ public InputStoreInfo(DataInputStore dataInputStore, int nFiles, long firstFileStart, long lastFileStart, long lastFileEnd) {
+ super();
+ this.dataInputStore = dataInputStore;
+ this.nFiles = nFiles;
+ this.firstFileStart = firstFileStart;
+ this.lastFileStart = lastFileStart;
+ this.lastFileEnd = lastFileEnd;
+ }
+
+ /**
+ * @return the nFiles
+ */
+ public int getnFiles() {
+ return nFiles;
+ }
+
+ /**
+ * @return the firstFileStart
+ */
+ public long getFirstFileStart() {
+ return firstFileStart;
+ }
+
+ /**
+ * @return the lastFileStart
+ */
+ public long getLastFileStart() {
+ return lastFileStart;
+ }
+
+ /**
+ * @return the lastFileEnd
+ */
+ public long getLastFileEnd() {
+ return lastFileEnd;
+ }
+
+ @Override
+ public String toString() {
+ return String.format("%s: %d files. First start %s, last start %s, last end %s", dataInputStore.getClass().getName(), nFiles,
+ PamCalendar.formatDBDateTime(firstFileStart), PamCalendar.formatDBDateTime(lastFileStart),
+ PamCalendar.formatDBDateTime(lastFileEnd));
+ }
+
+ /**
+ * @return the dataInputStore
+ */
+ public DataInputStore getDataInputStore() {
+ return dataInputStore;
+ }
+
+ /**
+ * Set the start times of all files in data set.
+ * @param allFileStarts
+ */
+ public void setFileStartTimes(long[] allFileStarts) {
+ this.fileStartTimes = allFileStarts;
+
+ }
+
+ /**
+ * @return the fileStartTimes
+ */
+ public long[] getFileStartTimes() {
+ return fileStartTimes;
+ }
+
+ /**
+ * @param firstFileStart the firstFileStart to set
+ */
+ public void setFirstFileStart(long firstFileStart) {
+ this.firstFileStart = firstFileStart;
+ }
+
+ /**
+ * @param lastFileStart the lastFileStart to set
+ */
+ public void setLastFileStart(long lastFileStart) {
+ this.lastFileStart = lastFileStart;
+ }
+
+ /**
+ * @param lastFileEnd the lastFileEnd to set
+ */
+ public void setLastFileEnd(long lastFileEnd) {
+ this.lastFileEnd = lastFileEnd;
+ }
+
+
+
+}
diff --git a/src/PamController/OfflineDataStore.java b/src/PamController/OfflineDataStore.java
index 86e2b367..52e64150 100644
--- a/src/PamController/OfflineDataStore.java
+++ b/src/PamController/OfflineDataStore.java
@@ -2,6 +2,7 @@ package PamController;
import java.awt.Window;
+import PamController.fileprocessing.StoreStatus;
import dataGram.DatagramManager;
import dataMap.OfflineDataMapPoint;
import pamScrollSystem.ViewLoadObserver;
@@ -52,7 +53,7 @@ public interface OfflineDataStore {
/**
* Moved this function over from binary data store.
- * Many storage systems may notbe able to do this, but some might !
+ * Many storage systems may not be able to do this, but some might !
* @param dataBlock
* @param dmp
* @return
@@ -63,4 +64,5 @@ public interface OfflineDataStore {
* @return the datagramManager
*/
public DatagramManager getDatagramManager();
+
}
diff --git a/src/PamController/PamController.java b/src/PamController/PamController.java
index c7546e04..444bb19a 100644
--- a/src/PamController/PamController.java
+++ b/src/PamController/PamController.java
@@ -55,10 +55,11 @@ import generalDatabase.DBControlUnit;
import javafx.application.Platform;
import javafx.stage.Stage;
import Array.ArrayManager;
-import PamController.command.MultiportController;
+import PamController.command.MulticastController;
import PamController.command.NetworkController;
import PamController.command.TerminalController;
import PamController.command.WatchdogComms;
+import PamController.fileprocessing.ReprocessManager;
import PamController.masterReference.MasterReferencePoint;
import PamController.settings.output.xml.PamguardXMLWriter;
import PamController.settings.output.xml.XMLWriterDialog;
@@ -119,6 +120,7 @@ public class PamController implements PamControllerInterface, PamSettings {
public static final int PAM_STALLED = 3;
public static final int PAM_INITIALISING = 4;
public static final int PAM_STOPPING = 5;
+ public static final int PAM_COMPLETE = 6;
// status' for RunMode = RUN_PAMVIEW
public static final int PAM_LOADINGDATA = 2;
@@ -256,8 +258,8 @@ public class PamController implements PamControllerInterface, PamSettings {
if (pamBuoyGlobals.getNetworkControlPort() != null) {
networkController = new NetworkController(this);
}
- if (pamBuoyGlobals.getMultiportAddress() != null) {
- new MultiportController(this);
+ if (pamBuoyGlobals.getMulticastAddress() != null) {
+ new MulticastController(this);
}
guiFrameManager = PamGUIManager.createGUI(this, object);
@@ -705,7 +707,7 @@ public class PamController implements PamControllerInterface, PamSettings {
Platform.exit();
// terminate the JVM
- System.exit(0);
+ System.exit(getPamStatus());
}
/**
@@ -1021,6 +1023,24 @@ public class PamController implements PamControllerInterface, PamSettings {
}
return foundUnits;
}
+
+ /**
+ * Get an Array list of PamControlledUnits of a particular class (exact matches only).
+ * @param unitClass PamControlledUnit class
+ * @return List of current instances of this class.
+ */
+ public ArrayList findControlledUnits(Class unitClass, boolean includeSubClasses) {
+ if (includeSubClasses == false) {
+ return findControlledUnits(unitClass);
+ }
+ ArrayList foundUnits = new ArrayList<>();
+ for (int i = 0; i < getNumControlledUnits(); i++) {
+ if (unitClass.isAssignableFrom(pamControlledUnits.get(i).getClass())) {
+ foundUnits.add(pamControlledUnits.get(i));
+ }
+ }
+ return foundUnits;
+ }
/**
* Check whether a controlled unit exists based on it's name.
@@ -1191,6 +1211,23 @@ public class PamController implements PamControllerInterface, PamSettings {
return false;
}
+ /*
+ *
+ * This needs to be called after prepareproces.
+ * Now we do some extra checks on the stores to see if we want to overwite data,
+ * carry on from where we left off, etc.
+ */
+ if (saveSettings && getRunMode() == RUN_NORMAL) { // only true on a button press or network start.
+ ReprocessManager reprocessManager = new ReprocessManager();
+ boolean goonthen = reprocessManager.checkOutputDataStatus();
+ if (goonthen == false) {
+ System.out.println("Data processing will not start since you've chosen not to overwrite existing output data");
+ pamStop();
+ setPamStatus(PAM_IDLE);
+ return false;
+ }
+ }
+
if (saveSettings) {
saveSettings(PamCalendar.getSessionStartTime());
}
@@ -1277,7 +1314,12 @@ public class PamController implements PamControllerInterface, PamSettings {
@Override
public void run() {
+ long t1 = System.currentTimeMillis();
while (checkRunStatus()) {
+ long t2 = System.currentTimeMillis();
+ if (t2 - t1 > 5000) {
+ System.out.printf("Stopping, but stuck in loop for CheckRunStatus for %3.1fs\n", (double) (t2-t1)/1000.);
+ }
try {
Thread.sleep(10);
} catch (InterruptedException e) {
diff --git a/src/PamController/PamSettingManager.java b/src/PamController/PamSettingManager.java
index e9119294..42488e1d 100644
--- a/src/PamController/PamSettingManager.java
+++ b/src/PamController/PamSettingManager.java
@@ -1752,10 +1752,10 @@ public class PamSettingManager {
fIs.close();
fOs.close();
} catch (FileNotFoundException e) {
- System.out.println(e.getMessage());
+ System.out.println("FileNotFound exception in PamSettingsManager: " + e.getMessage());
return false;
} catch (IOException e) {
- System.out.println(e.getMessage());
+ System.out.println("IO exception in PamSettingsManager: " + e.getMessage());
return false;
}
return true;
diff --git a/src/PamController/PamguardVersionInfo.java b/src/PamController/PamguardVersionInfo.java
index 072ba3de..791c56a3 100644
--- a/src/PamController/PamguardVersionInfo.java
+++ b/src/PamController/PamguardVersionInfo.java
@@ -31,12 +31,12 @@ public class PamguardVersionInfo {
* Version number, major version.minorversion.sub-release.
* Note: can't go higher than sub-release 'f'
*/
- static public final String version = "2.02.07b";
+ static public final String version = "2.02.07f";
/**
* Release date
*/
- static public final String date = "19 January 2023";
+ static public final String date = "4 April 2023";
// /**
// * Release type - Beta or Core
diff --git a/src/PamController/RawInputControlledUnit.java b/src/PamController/RawInputControlledUnit.java
new file mode 100644
index 00000000..8ac362a0
--- /dev/null
+++ b/src/PamController/RawInputControlledUnit.java
@@ -0,0 +1,21 @@
+package PamController;
+
+public abstract class RawInputControlledUnit extends PamControlledUnit {
+
+
+ public static final int RAW_INPUT_UNKNOWN = 0;
+ public static final int RAW_INPUT_FILEARCHIVE = 1;
+ public static final int RAW_INPUT_REALTIME = 2;
+
+ public RawInputControlledUnit(String unitType, String unitName) {
+ super(unitType, unitName);
+ }
+
+ /**
+ * Type of data input, which can be one of RAW_INPUT_UNKNOWN (0),
+ * RAW_INPUT_FILEARCHIVE (1), or RAW_INPUT_REALTIME (2)
+ * @return
+ */
+ public abstract int getRawInputType();
+
+}
diff --git a/src/PamController/command/BatchCommand.java b/src/PamController/command/BatchCommand.java
new file mode 100644
index 00000000..a3dadfa8
--- /dev/null
+++ b/src/PamController/command/BatchCommand.java
@@ -0,0 +1,64 @@
+package PamController.command;
+
+import networkTransfer.send.NetworkSender;
+import pamguard.GlobalArguments;
+
+public class BatchCommand extends ExtCommand {
+
+ public static final String commandId = "batchcommand";
+ private CommandManager commandManager;
+
+ public BatchCommand(CommandManager commandManager) {
+ super(commandId, true);
+ this.commandManager = commandManager;
+ }
+
+ @Override
+ public String execute(String command) {
+ /**
+ * this should have to identifiers. If they match the identifiers of
+ * this pamguard instance, then the command part is passed back to the manager
+ * otherwise it is ignored.
+ */
+ if (command == null) {
+ return null;
+ }
+ String[] bits = command.split(" ");
+ if (bits.length < 4) {
+ return null;
+ }
+ int id1 = 0;
+ int id2 = 0;
+ int expId1 = 0, expId2 = 0;
+ try {
+ id1 = Integer.valueOf(bits[1]);
+ id2 = Integer.valueOf(bits[2]);
+ }
+ catch (NumberFormatException e) {
+ System.out.println("Invalid BatchCommand: " + command);
+ return null;
+ }
+ String nid1 = GlobalArguments.getParam(NetworkSender.ID1);
+ String nid2 = GlobalArguments.getParam(NetworkSender.ID2);
+ if (bits[1].trim().equals(nid1) == false) {
+ return null;
+ }
+
+ // now trim the string to the end of the third comma and send on the rest
+ int comPos = -1;
+ for (int i = 0; i < 3; i++) {
+ comPos = command.indexOf(" ", comPos+1);
+ if (comPos < 0) {
+ return null;
+ }
+ }
+ String trueCommand = command.substring(comPos);
+ trueCommand = trueCommand.trim();
+// System.out.printf(">>>>>>>>>>>>>>>>>>> Batchcommand execute \"%s\" in command manager %s\n", trueCommand, commandManager.getClass().getName());
+ if (commandManager.interpretCommand(trueCommand)) {
+// return commandManager.get
+ }
+ return null;
+ }
+
+}
diff --git a/src/PamController/command/BatchStatusCommand.java b/src/PamController/command/BatchStatusCommand.java
new file mode 100644
index 00000000..a58012b2
--- /dev/null
+++ b/src/PamController/command/BatchStatusCommand.java
@@ -0,0 +1,31 @@
+package PamController.command;
+
+import Acquisition.AcquisitionControl;
+import PamController.PamController;
+import pamViewFX.PamControlledGUIFX;
+
+/**
+ * Command to get PAMGuard to send back the batch processing status.
+ *
+ * @author dg50
+ *
+ */
+public class BatchStatusCommand extends ExtCommand {
+
+ public static final String commandId = "batchstatus";
+
+ public BatchStatusCommand() {
+ super(commandId, true);
+ // TODO Auto-generated constructor stub
+ }
+
+ @Override
+ public String execute(String command) {
+ AcquisitionControl daqControl = (AcquisitionControl) PamController.getInstance().findControlledUnit(AcquisitionControl.class, null);
+ if (daqControl == null) {
+ return null;
+ }
+ return daqControl.getBatchStatus();
+ }
+
+}
diff --git a/src/PamController/command/CommandManager.java b/src/PamController/command/CommandManager.java
index 01533e28..87f4b888 100644
--- a/src/PamController/command/CommandManager.java
+++ b/src/PamController/command/CommandManager.java
@@ -40,6 +40,8 @@ public abstract class CommandManager extends PamControlledUnit {
commandsList.add(new HelpCommand(this));
commandsList.add(new GetXMLSettings());
commandsList.add(new SetXMLSettings());
+ commandsList.add(new BatchStatusCommand());
+ commandsList.add(new BatchCommand(this));
}
@@ -81,16 +83,16 @@ public abstract class CommandManager extends PamControlledUnit {
}
ExtCommand extCommand = findCommand(command);
if (extCommand == null) {
- sendData("Cmd \"" + commandString + "\" Not Recognised.");
+ sendData(extCommand, "Cmd \"" + commandString + "\" Not Recognised.");
return false;
}
if (extCommand.canExecute() == false) {
- sendData("Cmd \"" + command + "\" Cannot Execute.");
+ sendData(extCommand, "Cmd \"" + command + "\" Cannot Execute.");
// sendData(" Cmd return string = " + extCommand.getReturnString());
return false;
}
String output = extCommand.executeCommand(commandString);
- sendData(output);
+ sendData(extCommand,output);
return true;
@@ -154,9 +156,10 @@ public abstract class CommandManager extends PamControlledUnit {
/**
* Reply to data called from InterpredData
+ * @param extCommand
* @param dataString
* @return true if replay successful
*/
- abstract public boolean sendData(String dataString);
+ abstract public boolean sendData(ExtCommand extCommand, String dataString);
}
diff --git a/src/PamController/command/ExitCommand.java b/src/PamController/command/ExitCommand.java
index a44c91ce..f12a6dd6 100644
--- a/src/PamController/command/ExitCommand.java
+++ b/src/PamController/command/ExitCommand.java
@@ -10,15 +10,20 @@ import PamController.PamSettingManager;
*/
public class ExitCommand extends ExtCommand {
+ public static final String commandId = "Exit";
+
public ExitCommand() {
- super("Exit", false);
+ super(commandId, false);
}
@Override
public String execute(String command) {
PamController.getInstance().pamStop();
PamSettingManager.getInstance().saveFinalSettings();
- System.exit(0);
+ PamController pamController = PamController.getInstance();
+ pamController.pamClose();
+ // shut down the JavaFX thread and the JVM
+ pamController.shutDownPamguard();
return getName();
}
diff --git a/src/PamController/command/HelpCommand.java b/src/PamController/command/HelpCommand.java
index 400516ad..3350c828 100644
--- a/src/PamController/command/HelpCommand.java
+++ b/src/PamController/command/HelpCommand.java
@@ -25,7 +25,7 @@ public class HelpCommand extends ExtCommand {
out += "\n";
}
}
- commandManager.sendData(out);
+ commandManager.sendData(this, out);
return getName();
}
diff --git a/src/PamController/command/MultiportController.java b/src/PamController/command/MulticastController.java
similarity index 65%
rename from src/PamController/command/MultiportController.java
rename to src/PamController/command/MulticastController.java
index ead9e397..de92b6ae 100644
--- a/src/PamController/command/MultiportController.java
+++ b/src/PamController/command/MulticastController.java
@@ -12,12 +12,14 @@ import java.nio.charset.StandardCharsets;
import PamController.PamController;
import PamController.pamBuoyGlobals;
+import networkTransfer.send.NetworkSender;
+import pamguard.GlobalArguments;
-public class MultiportController extends CommandManager {
+public class MulticastController extends CommandManager {
//The multicast addresses are in the range 224.0.0.0 through 239.255.255.255
- private static String unitName = "Multiport Controller";
+ private static String unitName = "Multicast Controller";
private PamController pamController;
private String mAddress;
private int mPort;
@@ -29,11 +31,11 @@ public class MultiportController extends CommandManager {
private byte[] byteBuffer = new byte[MAX_COMMAND_LENGTH];
private DatagramPacket lastDatagram;
- public MultiportController(PamController pamController) {
+ public MulticastController(PamController pamController) {
super(pamController, unitName);
this.pamController = pamController;
- this.mAddress = pamBuoyGlobals.getMultiportAddress();
- this.mPort = pamBuoyGlobals.getMuliportPort();
+ this.mAddress = pamBuoyGlobals.getMulticastAddress();
+ this.mPort = pamBuoyGlobals.getMulticastPort();
Thread t = new Thread(new ListenerThread());
t.start();
@@ -83,17 +85,35 @@ public class MultiportController extends CommandManager {
private void processDatagram(DatagramPacket datagram) {
lastDatagram = datagram;
String str = new String(datagram.getData(), 0, datagram.getLength());
-// str = str.substring(0, datagram.getLength());
- System.out.println("Datagram received \"" + str + "\"");
+ str = str.substring(0, datagram.getLength());
+// System.out.println("Datagram received \"" + str + "\"");
interpretCommand(str);
}
@Override
- public boolean sendData(String dataString) {
+ public boolean sendData(ExtCommand extCommand, String dataString) {
+ if (dataString == null || dataString.length() == 0) {
+ return false;
+ }
DatagramPacket senderInfo = lastDatagram;
- System.out.printf("Send back data \"%s\" to %s port %d\n", dataString, senderInfo.getAddress(), senderInfo.getPort());
+ String commandName;
+ if (extCommand == null) {
+ commandName = "Unknown";
+ }
+ else {
+ commandName = extCommand.getName();
+ }
+// System.out.printf("Send back data \"%s\" to %s port %d\n", dataString, senderInfo.getAddress(), senderInfo.getPort());
+ /*
+ * for multicast, we need to send a slightly different string back which has the station id as part of
+ * the returned string. These will be 0 if they weren't passed in at the command line.
+ */
+ String id1 = GlobalArguments.getParam(NetworkSender.ID1);
+ String id2 = GlobalArguments.getParam(NetworkSender.ID2);
+ String bigString = String.format("%s,%s,%s,%s", commandName, id1, id2, dataString);
+
// dataString += "\n";
- DatagramPacket packet = new DatagramPacket(dataString.getBytes(), dataString.length());
+ DatagramPacket packet = new DatagramPacket(bigString.getBytes(), bigString.length());
packet.setAddress(senderInfo.getAddress());
packet.setPort(senderInfo.getPort());
try {
diff --git a/src/PamController/command/NetworkController.java b/src/PamController/command/NetworkController.java
index 2090851b..82268bd1 100644
--- a/src/PamController/command/NetworkController.java
+++ b/src/PamController/command/NetworkController.java
@@ -124,16 +124,14 @@ public class NetworkController extends CommandManager {
}
}
-
- public boolean sendData(String dataString) {
+ @Override
+ public boolean sendData(ExtCommand extCommand, String dataString) {
DatagramPacket packet = new DatagramPacket(dataString.getBytes(), dataString.length());
packet.setAddress(udpPacket.getAddress());
packet.setPort(udpPacket.getPort());
try {
receiveSocket.send(packet);
-// receiveSocket.
} catch (IOException e) {
- // TODO Auto-generated catch block
e.printStackTrace();
return false;
}
diff --git a/src/PamController/command/TerminalController.java b/src/PamController/command/TerminalController.java
index e8589cd2..533b23ee 100644
--- a/src/PamController/command/TerminalController.java
+++ b/src/PamController/command/TerminalController.java
@@ -17,7 +17,7 @@ public class TerminalController extends CommandManager {
}
@Override
- public boolean sendData(String dataString) {
+ public boolean sendData(ExtCommand extcommand, String dataString) {
System.out.println(dataString);
return true;
}
diff --git a/src/PamController/extern/ExternalCapture.java b/src/PamController/extern/ExternalCapture.java
index a4ff2259..8413467f 100644
--- a/src/PamController/extern/ExternalCapture.java
+++ b/src/PamController/extern/ExternalCapture.java
@@ -46,7 +46,7 @@ public class ExternalCapture extends ExternalController {
}
}
} catch (IOException e) {
- System.out.println(e.getMessage());
+ System.out.println("IOError in ExternalCapture.readInputStream: " + e.getMessage());
}
}
diff --git a/src/PamController/fileprocessing/ReprocessChoiceDialog.java b/src/PamController/fileprocessing/ReprocessChoiceDialog.java
new file mode 100644
index 00000000..f2b9eee5
--- /dev/null
+++ b/src/PamController/fileprocessing/ReprocessChoiceDialog.java
@@ -0,0 +1,116 @@
+package PamController.fileprocessing;
+
+import java.awt.BorderLayout;
+import java.awt.GridBagConstraints;
+import java.awt.GridBagLayout;
+import java.awt.Window;
+import java.util.List;
+
+import javax.swing.ButtonGroup;
+import javax.swing.JLabel;
+import javax.swing.JPanel;
+import javax.swing.JRadioButton;
+import javax.swing.border.TitledBorder;
+
+import PamUtils.PamCalendar;
+import PamView.dialog.PamDialog;
+import PamView.dialog.PamGridBagContraints;
+import PamView.dialog.warn.WarnOnce;
+import PamView.panel.PamAlignmentPanel;
+
+public class ReprocessChoiceDialog extends PamDialog {
+
+ private static final long serialVersionUID = 1L;
+
+ private static ReprocessChoiceDialog singleInstance;
+
+ private JRadioButton[] choiceButtons;
+
+ private ReprocessStoreChoice chosenChoice = null;
+
+ private StoreChoiceSummary choiceSummary;
+
+ private ReprocessChoiceDialog(Window parentFrame, StoreChoiceSummary choiceSummary) {
+ super(parentFrame, "Existing Output Data", false);
+ this.choiceSummary = choiceSummary;
+ JPanel mainPanel = new JPanel(new BorderLayout());
+ JPanel infoPanel = new JPanel(new GridBagLayout());
+ infoPanel.setBorder(new TitledBorder("Data Summary"));
+ mainPanel.add(infoPanel, BorderLayout.NORTH);
+ GridBagConstraints c = new PamGridBagContraints();
+ c.gridx = c.gridy = 0;
+ String inStr = String.format("Input data dates: %s to %s", PamCalendar.formatDBDateTime(choiceSummary.getInputStartTime()),
+ PamCalendar.formatDBDateTime(choiceSummary.getInputEndTime()));
+ infoPanel.add(new JLabel(inStr), c);
+ c.gridy++;
+ String outStr = String.format("Output data dates: %s to %s", PamCalendar.formatDBDateTime(choiceSummary.getOutputStartTime()),
+ PamCalendar.formatDBDateTime(choiceSummary.getOutputEndTime()));
+ infoPanel.add(new JLabel(outStr), c);
+ String stateStr;
+ if (choiceSummary.isProcessingComplete()) {
+ stateStr = "Processing appears to be complete";
+ }
+ else {
+ stateStr = "Processing appears to be partially complete";
+ }
+ c.gridy++;
+ infoPanel.add(new JLabel(stateStr), c);
+
+
+ JPanel choicePanel = new PamAlignmentPanel(new GridBagLayout(), BorderLayout.WEST);
+ choicePanel.setBorder(new TitledBorder("Chose what to do"));
+ c = new PamGridBagContraints();
+ mainPanel.add(BorderLayout.SOUTH, choicePanel);
+ List userChoices = choiceSummary.getChoices();
+ choiceButtons = new JRadioButton[userChoices.size()];
+ ButtonGroup bg = new ButtonGroup();
+ for (int i = 0; i < userChoices.size(); i++) {
+ ReprocessStoreChoice aChoice = userChoices.get(i);
+ choiceButtons[i] = new JRadioButton(aChoice.toString());
+ choiceButtons[i].setToolTipText(aChoice.getToolTip());
+ choicePanel.add(choiceButtons[i], c);
+ c.gridy++;
+ }
+ setDialogComponent(mainPanel);
+ getCancelButton().setVisible(false);
+ }
+
+ public static ReprocessStoreChoice showDialog(Window parentFrame, StoreChoiceSummary choices) {
+// if (singleInstance == null || singleInstance.getOwner() != parentFrame) {
+ singleInstance = new ReprocessChoiceDialog(parentFrame, choices);
+// }
+ singleInstance.setVisible(true);
+ return singleInstance.chosenChoice;
+ }
+
+ @Override
+ public boolean getParams() {
+ List userChoices = choiceSummary.getChoices();
+ for (int i = 0; i < choiceButtons.length; i++) {
+ if (choiceButtons[i].isSelected()) {
+ chosenChoice = userChoices.get(i);
+ break;
+ }
+ }
+ if (chosenChoice == ReprocessStoreChoice.OVERWRITEALL) {
+ String w = "Are you sure you want to delete / overwrite all existing output data ?";
+ int ans = WarnOnce.showWarning("Overwrite existing data", w, WarnOnce.OK_CANCEL_OPTION);
+ if (ans == WarnOnce.CANCEL_OPTION) {
+ return false;
+ }
+ }
+ return chosenChoice != null;
+ }
+
+ @Override
+ public void cancelButtonPressed() {
+ chosenChoice = null;
+ }
+
+ @Override
+ public void restoreDefaultSettings() {
+ // TODO Auto-generated method stub
+
+ }
+
+}
diff --git a/src/PamController/fileprocessing/ReprocessManager.java b/src/PamController/fileprocessing/ReprocessManager.java
new file mode 100644
index 00000000..4f05913a
--- /dev/null
+++ b/src/PamController/fileprocessing/ReprocessManager.java
@@ -0,0 +1,285 @@
+package PamController.fileprocessing;
+
+import java.util.ArrayList;
+
+import PamController.DataInputStore;
+import PamController.DataOutputStore;
+import PamController.InputStoreInfo;
+import PamController.OfflineDataStore;
+import PamController.PamControlledUnit;
+import PamController.PamController;
+import PamController.PamGUIManager;
+import PamController.RawInputControlledUnit;
+import PamUtils.PamCalendar;
+import PamView.dialog.warn.WarnOnce;
+import pamguard.GlobalArguments;
+
+/**
+ * Set of functions to help decide what to do when reprocessing.
+ * These are probably all called from AcquisitionProcess, but it's easier to have them in their own class.
+ * @author dg50
+ *
+ */
+public class ReprocessManager {
+
+ /**
+ public ReprocessManager() {
+ // TODO Auto-generated constructor stub
+ }
+
+ /*
+ * run checks on the output data storage system. If data already exist in the output
+ * we may not want to start again.
+ */
+ public boolean checkOutputDataStatus() {
+
+ StoreChoiceSummary choiceSummary = null;
+ if (isOfflineFiles()) {
+ choiceSummary = checkIOFilesStatus();
+ }
+ else {
+ /*
+ * don't really need to do anything for real time processing since adding
+ * more data to existing stores is normal behaviour.
+ */
+ return true;
+ }
+ if (choiceSummary == null) {
+ return true;
+ }
+
+ if (choiceSummary.getInputStoreInfo() == null) {
+ return true;
+ }
+
+ // need to decide what to do based on the list of possible choices.
+ ReprocessStoreChoice choice = chosePartStoreAction(choiceSummary);
+
+ /**
+ * Need to call this even though we aren't reprocessing so that
+ * the Folderinput stream reports correctly on how many files have
+ * been processed.
+ */
+ boolean setupOK = setupInputStream(choiceSummary, choice);
+
+ if (choice == ReprocessStoreChoice.DONTSSTART) {
+ return false;
+ }
+
+ boolean deleteOK = deleteOldData(choiceSummary, choice);
+
+ return true;
+
+ }
+
+
+ private boolean setupInputStream(StoreChoiceSummary choiceSummary, ReprocessStoreChoice choice) {
+ // work out the first file index and send it to the appropriate input module.
+ long deleteFrom = getDeleteFromTime(choiceSummary, choice);
+ ArrayList inputStores = PamController.getInstance().findControlledUnits(DataInputStore.class, true);
+ if (inputStores == null || inputStores.size() == 0) {
+ return false;
+ }
+ InputStoreInfo inputInfo = null;
+ boolean OK = true;
+ for (PamControlledUnit aPCU : inputStores) {
+ DataInputStore inputStore = (DataInputStore) aPCU;
+ OK &= inputStore.setAnalysisStartTime(deleteFrom);
+// System.out.println("Input store info: " + inputInfo);
+ }
+ return OK;
+ }
+
+
+ /**
+ * Just gets on and does it. The user should already have been asked what they
+ * want to do, so don't ask again.
+ * @param choiceSummary
+ * @param choice
+ */
+ private boolean deleteOldData(StoreChoiceSummary choiceSummary, ReprocessStoreChoice choice) {
+ long deleteFrom = getDeleteFromTime(choiceSummary, choice);
+ // go through the data stores and tell them to delete from that time.
+ if (deleteFrom == Long.MAX_VALUE) {
+ return false;
+ }
+ ArrayList outputStores = PamController.getInstance().findControlledUnits(DataOutputStore.class, true);
+ boolean partStores = false;
+ boolean ok = true;
+ for (PamControlledUnit aPCU : outputStores) {
+ DataOutputStore offlineStore = (DataOutputStore) aPCU;
+ ok &= offlineStore.deleteDataFrom(deleteFrom);
+ }
+ return ok;
+ }
+
+
+ private long getDeleteFromTime(StoreChoiceSummary choiceSummary, ReprocessStoreChoice choice) {
+ if (choice == null) {
+ return Long.MAX_VALUE; // I don't think this can happen, but you never know.
+ }
+ Long t = null;
+ switch (choice) {
+ case CONTINUECURRENTFILE:
+ t = choiceSummary.getInputTimeForIndex(choiceSummary.getFileIndexBefore(choiceSummary.getOutputEndTime()));
+ break;
+ case CONTINUENEXTFILE:
+ t = choiceSummary.getInputTimeForIndex(choiceSummary.getFileIndexAfter(choiceSummary.getOutputEndTime()));
+ break;
+ case DONTSSTART: // we should'nt get here with this option.
+ return Long.MAX_VALUE;
+ case OVERWRITEALL:
+ return 0; // delete from start. (
+ case STARTNORMAL: // we should'nt get here with this option.
+ return Long.MAX_VALUE;
+ default:
+ break;
+
+ }
+ if (t == null) {
+ // shouldn't happen, don't do any deleteing
+ return Long.MAX_VALUE;
+ }
+ else {
+ return t;
+ }
+ }
+
+
+ /**
+ * Check the output of current files and databases and return a flag to PamController saying whether or
+ * not processing should actually start, possibly overwriting, or if we need to not start to avoid overwriting.
+ * @return true if processing should start.
+ */
+ private StoreChoiceSummary checkIOFilesStatus() {
+ /**
+ * Get information about the input.
+ *
+ */
+ ArrayList inputStores = PamController.getInstance().findControlledUnits(DataInputStore.class, true);
+ if (inputStores == null || inputStores.size() == 0) {
+ return new StoreChoiceSummary(null, ReprocessStoreChoice.STARTNORMAL);
+ }
+ InputStoreInfo inputInfo = null;
+ for (PamControlledUnit aPCU : inputStores) {
+ DataInputStore inputStore = (DataInputStore) aPCU;
+ inputInfo = inputStore.getStoreInfo(true);
+// System.out.println("Input store info: " + inputInfo);
+ }
+ StoreChoiceSummary choiceSummary = new StoreChoiceSummary(inputInfo);
+
+ if (inputInfo == null || inputInfo.getFileStartTimes() == null) {
+ choiceSummary.addChoice(ReprocessStoreChoice.STARTNORMAL);
+ return choiceSummary;
+ }
+
+ ArrayList outputStores = PamController.getInstance().findControlledUnits(DataOutputStore.class, true);
+ boolean partStores = false;
+ int nOutputStores = 0;
+ for (PamControlledUnit aPCU : outputStores) {
+ DataOutputStore offlineStore = (DataOutputStore) aPCU;
+ StoreStatus status = offlineStore.getStoreStatus(false);
+ nOutputStores++;
+ if (status == null) {
+ continue;
+ }
+ if (status.getStoreStatus() == StoreStatus.STATUS_HASDATA) {
+ status = offlineStore.getStoreStatus(true); // get more detail.
+ partStores = true;
+ System.out.printf("Storage %s already contains some data\n", offlineStore.getDataSourceName());
+ choiceSummary.testOutputEndTime(status.getLastDataTime());
+ choiceSummary.testOutputStartTime(status.getFirstDataTime());
+ }
+ }
+
+ if (partStores == false) {
+// choiceSummary.addChoice(ReprocessStoreChoice.STARTNORMAL);
+ return null; // no part full stores, so can start without questions
+ }
+ if (choiceSummary.getInputStartTime() >= choiceSummary.getOutputEndTime()) {
+ /*
+ * looks like it's new data that starts after the end of the current store,
+ * so there is no need to do anything.
+ */
+ choiceSummary.addChoice(ReprocessStoreChoice.STARTNORMAL);
+ return choiceSummary;
+ }
+ /*
+ * If we land here, it looks like we have overlapping data. so need to make a decision
+ * First, check to see if processing has actually completed which will be the case if
+ * the data time and the end of the files are the same.
+ */
+ choiceSummary.addChoice(ReprocessStoreChoice.DONTSSTART);
+ choiceSummary.addChoice(ReprocessStoreChoice.OVERWRITEALL);
+ if (choiceSummary.isProcessingComplete() == false) {
+ choiceSummary.addChoice(ReprocessStoreChoice.CONTINUECURRENTFILE);
+ choiceSummary.addChoice(ReprocessStoreChoice.CONTINUENEXTFILE);
+ }
+
+ return choiceSummary;
+
+ }
+
+ /**
+ * Either opens a dialog to ask the user, or uses a choice entered into the command line for nogui mode.
+ * Decide what to do with stores that already have data. Can return continue from end or overwrite
+ * in which case stores will be deleted and we'll start again. The chosen action will need to be
+ * communicated to the various inputs.
+ * @param choices
+ */
+ private ReprocessStoreChoice chosePartStoreAction(StoreChoiceSummary choices) {
+ /**
+ * Do we really have to deal with multiple inputs ? Can I envisage a situation where there is
+ * ever more than one input going at any one time ? not really, but should I add code
+ * to make sure that there really can be only one ? i.e. two daq's would be allowed for real
+ * time processing, but only one for offline ? could do all I guess by looking at sources of
+ * all output data blocks and doing it on a case by case basis. All we have to do here though
+ * is to get an answer about what to do.
+ */
+ // see if we've got a global parameter passed in as an argument
+ String arg = GlobalArguments.getParam(ReprocessStoreChoice.paramName);
+ if (arg != null) {
+ ReprocessStoreChoice choice = ReprocessStoreChoice.valueOf(arg);
+ if (choice == null) {
+ String warn = String.format("Reprocessing storage input parameter %s value \"%s\" is not a recognised value", ReprocessStoreChoice.paramName, arg);
+ WarnOnce.showWarning("Invalid input parameter", warn, WarnOnce.WARNING_MESSAGE);
+ }
+ if (choice == ReprocessStoreChoice.CONTINUECURRENTFILE || choice == ReprocessStoreChoice.CONTINUENEXTFILE) {
+ if (choices.isProcessingComplete()) {
+ return ReprocessStoreChoice.DONTSSTART;
+ }
+ }
+ return choice;
+ }
+ if (PamGUIManager.getGUIType() == PamGUIManager.NOGUI) {
+ System.out.println("In Nogui mode you should set a choice as to how to handle existing storage overwrites. Using default of overwriting everything");
+ return ReprocessStoreChoice.OVERWRITEALL;
+ }
+
+ // otherwise we'll need to show a dialog to let the user decide what to do
+ ReprocessStoreChoice choice = ReprocessChoiceDialog.showDialog(PamController.getMainFrame(), choices);
+
+ return choice;
+ }
+
+ /**
+ * Return true if we seem to be reprocessing offline files.
+ * Note that this might be the Tritech data as well as the sound acquisition so
+ * have added an abstract intermediate class on the controlled units so we can check them all.
+ * @return
+ */
+ public boolean isOfflineFiles() {
+ ArrayList sources = PamController.getInstance().findControlledUnits(RawInputControlledUnit.class, true);
+ if (sources == null) {
+ return false;
+ }
+ for (PamControlledUnit pcu : sources) {
+ RawInputControlledUnit rawPCU = (RawInputControlledUnit) pcu;
+ if (rawPCU.getRawInputType() == RawInputControlledUnit.RAW_INPUT_FILEARCHIVE) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+}
diff --git a/src/PamController/fileprocessing/ReprocessStoreChoice.java b/src/PamController/fileprocessing/ReprocessStoreChoice.java
new file mode 100644
index 00000000..6a6bd019
--- /dev/null
+++ b/src/PamController/fileprocessing/ReprocessStoreChoice.java
@@ -0,0 +1,53 @@
+package PamController.fileprocessing;
+
+/**
+ * Choices on what to do when re-processing data and finding that output data already exist.
+ * @author dg50
+ *
+ */
+public enum ReprocessStoreChoice {
+
+
+ STARTNORMAL, CONTINUECURRENTFILE, CONTINUENEXTFILE, OVERWRITEALL, DONTSSTART;
+
+ public static final String paramName = "-reprocessoption";
+
+ @Override
+ public String toString() {
+ switch (this) {
+ case STARTNORMAL:
+ return "Start normally. Note risk of overwriting!";
+ case CONTINUECURRENTFILE:
+ return "Continue from start of last input file processed";
+ case CONTINUENEXTFILE:
+ return "Continue from start of next input file to process";
+ case DONTSSTART:
+ return "Don't start processing";
+ case OVERWRITEALL:
+ return "Overwrite existing output data";
+ default:
+ break;
+ }
+ return null;
+ }
+
+ public String getToolTip() {
+ switch (this) {
+ case STARTNORMAL:
+ return "No risk of data overlap, so system will start normally";
+ case CONTINUECURRENTFILE:
+ return "System will work out how far data processing has got and continue from the start of the file it stopped in";
+ case CONTINUENEXTFILE:
+ return "System will work out how far data processing has got and continue from the start of the file AFTER the one it stopped in";
+ case DONTSSTART:
+ return "Processing will not start. Select alternative storage locations / databases and try again";
+ case OVERWRITEALL:
+ return "Overwrite existing output data. Existing data will be deleted";
+ default:
+ break;
+ }
+ return null;
+ }
+
+
+}
diff --git a/src/PamController/fileprocessing/StoreChoiceSummary.java b/src/PamController/fileprocessing/StoreChoiceSummary.java
new file mode 100644
index 00000000..f0f13c3a
--- /dev/null
+++ b/src/PamController/fileprocessing/StoreChoiceSummary.java
@@ -0,0 +1,234 @@
+package PamController.fileprocessing;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import PamController.InputStoreInfo;
+
+/**
+ * Summary information about the data stores.
+ * @author dg50
+ *
+ */
+public class StoreChoiceSummary {
+
+ private long outputEndTime;
+
+ private long outputStartTime;
+
+ private List choices = new ArrayList<>();
+
+ private InputStoreInfo inputStoreInfo;
+
+ public StoreChoiceSummary(InputStoreInfo info, ReprocessStoreChoice singleChoice) {
+ this.inputStoreInfo = info;
+ addChoice(singleChoice);
+ }
+
+ public StoreChoiceSummary(long outputEndTime, InputStoreInfo inputStoreInfo) {
+ super();
+ this.outputEndTime = outputEndTime;
+ this.inputStoreInfo = inputStoreInfo;
+ }
+
+ public StoreChoiceSummary(InputStoreInfo inputInfo) {
+ this.inputStoreInfo = inputInfo;
+ }
+
+ /**
+ * Get the number of choices. If it's only one, then there
+ * isn't a lot to do. If it's >1, then need a decision in the
+ * form of a command line instruction or a user dialog.
+ * @return number of choices.
+ */
+ public int getNumChoices() {
+ return choices.size();
+ }
+
+ /**
+ * Is processing complete, i.e. last time in output matches last time
+ * in input data.
+ * @return true if processing appears to be complete.
+ */
+ public boolean isProcessingComplete() {
+ if (inputStoreInfo == null) {
+ return false;
+ }
+ long inputEnd = getInputEndTime();
+ long outputEnd = getOutputEndTime();
+ long diff = inputEnd-outputEnd;
+ return (diff < 1000);
+ }
+
+ /**
+ * Add a reasonable choice to what the user can select to do.
+ * @param choice
+ */
+ public void addChoice(ReprocessStoreChoice choice) {
+ choices.add(choice);
+ }
+
+ /**
+ * @return the start time of the first input file
+ */
+ public Long getInputStartTime() {
+ if (inputStoreInfo == null) {
+ return null;
+ }
+ return inputStoreInfo.getFirstFileStart();
+ }
+
+ /**
+ * @return the start time of the first input file
+ */
+ public Long getInputEndTime() {
+ if (inputStoreInfo == null) {
+ return null;
+ }
+ return inputStoreInfo.getLastFileEnd();
+ }
+
+ /**
+ * @return the outputEndTime
+ */
+ public long getOutputEndTime() {
+ return outputEndTime;
+ }
+
+
+ /**
+ * Set the last data time, but only if the passed value
+ * is not null and is bigger than the current value.
+ * @param lastDataTime
+ * @return largest of current and passed value.
+ */
+ public long testOutputEndTime(Long lastDataTime) {
+ if (lastDataTime == null) {
+ return this.getOutputEndTime();
+ }
+ setOutputEndTime(Math.max(outputEndTime, lastDataTime));
+ return getOutputEndTime();
+ }
+
+ /**
+ * Set the last data time, but only if the passed value
+ * is not null and is bigger than the current value.
+ * @param lastDataTime
+ * @return largest of current and passed value.
+ */
+ public long testOutputStartTime(Long firstDataTime) {
+ if (firstDataTime == null) {
+ return this.getOutputStartTime();
+ }
+ if (outputStartTime == 0 || firstDataTime < outputStartTime) {
+ outputStartTime = firstDataTime;
+ }
+ return getOutputStartTime();
+ }
+
+ /**
+ * @param outputEndTime the outputEndTime to set
+ */
+ public void setOutputEndTime(long outputEndTime) {
+ this.outputEndTime = outputEndTime;
+ }
+
+ /**
+ * @return the inputStoreInfo
+ */
+ public InputStoreInfo getInputStoreInfo() {
+ return inputStoreInfo;
+ }
+
+ /**
+ * @param inputStoreInfo the inputStoreInfo to set
+ */
+ public void setInputStoreInfo(InputStoreInfo inputStoreInfo) {
+ this.inputStoreInfo = inputStoreInfo;
+ }
+
+ /**
+ * @return the choices
+ */
+ public List getChoices() {
+ return choices;
+ }
+
+ /**
+ * @return the outputStartTime
+ */
+ public long getOutputStartTime() {
+ return outputStartTime;
+ }
+
+ /**
+ * @param outputStartTime the outputStartTime to set
+ */
+ public void setOutputStartTime(long outputStartTime) {
+ this.outputStartTime = outputStartTime;
+ }
+
+ /**
+ * Get the index of the file that starts before or exactly at the given time.
+ * @param inputEndTime
+ * @return index of file, or -1 if none found.
+ */
+ public int getFileIndexBefore(Long inputEndTime) {
+ if (inputStoreInfo == null) {
+ return -1;
+ }
+ long[] fileStarts = inputStoreInfo.getFileStartTimes();
+ if (fileStarts == null) {
+ return -1;
+ }
+ for (int i = fileStarts.length-1; i>= 0; i--) {
+ if (fileStarts[i] <= inputEndTime) {
+ return i;
+ }
+ }
+ return -1;
+ }
+
+ /**
+ * Get the start time in millis of a file for the given index.
+ * @param fileIndex
+ * @return file time, or null if no file available.
+ */
+ public Long getInputTimeForIndex(int fileIndex) {
+ if (inputStoreInfo == null) {
+ return null;
+ }
+ long[] fileStarts = inputStoreInfo.getFileStartTimes();
+ if (fileStarts == null) {
+ return null;
+ }
+ if (fileIndex < 0 || fileIndex >= fileStarts.length) {
+ return null;
+ }
+ return fileStarts[fileIndex];
+ }
+
+ /**
+ * Get the index of the file that starts after the given time.
+ * @param inputEndTime
+ * @return index of file, or -1 if none found.
+ */
+ public int getFileIndexAfter(Long inputEndTime) {
+ if (inputStoreInfo == null) {
+ return -1;
+ }
+ long[] fileStarts = inputStoreInfo.getFileStartTimes();
+ if (fileStarts == null) {
+ return -1;
+ }
+ for (int i = 0; i < fileStarts.length; i++) {
+ if (fileStarts[i] > inputEndTime) {
+ return i;
+ }
+ }
+ return -1;
+ }
+
+
+
+}
diff --git a/src/PamController/fileprocessing/StoreStatus.java b/src/PamController/fileprocessing/StoreStatus.java
new file mode 100644
index 00000000..5f50fbc1
--- /dev/null
+++ b/src/PamController/fileprocessing/StoreStatus.java
@@ -0,0 +1,115 @@
+package PamController.fileprocessing;
+
+import java.io.File;
+
+import PamController.OfflineDataStore;
+
+/**
+ * Class to carry information about an OfflineDataStore. Used when restarting offline
+ * processing to help work out if we should overwrite, start again, etc.
+ * @author dg50
+ *
+ */
+abstract public class StoreStatus {
+
+ public static final int STATUS_MISSING = 1;
+
+ public static final int STATUS_EMPTY = 2;
+
+ public static final int STATUS_HASDATA = 3;
+
+ private OfflineDataStore offlineDataStore;
+
+ /**
+ * Time of first data, may be null if detail not asked for or if
+ * hasData is false.
+ */
+ private Long firstDataTime;
+
+ /**
+ * Time of last data, may be null if detail not asked for or if
+ * hasData is false.
+ */
+ private Long lastDataTime;
+
+ /**
+ * General status flag.
+ */
+ private int storeStatus;
+
+ public StoreStatus(OfflineDataStore offlineDataStore) {
+ this.offlineDataStore = offlineDataStore;
+ }
+
+ /**
+ * Get the amount of free space for this storage.
+ * @return free space in bytes.
+ */
+ public abstract long getFreeSpace();
+
+ public long getFreeSpace(String currDir) {
+ if (currDir == null) {
+ return 0;
+ }
+ File dirFile = new File(currDir);
+ long space = 0;
+ try {
+ space = dirFile.getUsableSpace();
+ }
+ catch (SecurityException e) {
+ System.out.printf("Security exception getting space for %s: \n%s\n", currDir, e.getMessage());
+ }
+ return space;
+ }
+
+ /**
+ * @return the firstDataTime
+ */
+ public Long getFirstDataTime() {
+ return firstDataTime;
+ }
+
+ /**
+ * @param firstDataTime the firstDataTime to set
+ */
+ public void setFirstDataTime(Long firstDataTime) {
+ this.firstDataTime = firstDataTime;
+ }
+
+ /**
+ * @return the lastDataTime
+ */
+ public Long getLastDataTime() {
+ return lastDataTime;
+ }
+
+ /**
+ * @param lastDataTime the lastDataTime to set
+ */
+ public void setLastDataTime(Long lastDataTime) {
+ this.lastDataTime = lastDataTime;
+ }
+
+ /**
+ * @return the storeStatus
+ */
+ public int getStoreStatus() {
+ return storeStatus;
+ }
+
+ /**
+ * @param storeStatus the storeStatus to set
+ */
+ public void setStoreStatus(int storeStatus) {
+ this.storeStatus = storeStatus;
+ }
+
+ /**
+ * @return the offlineDataStore
+ */
+ public OfflineDataStore getOfflineDataStore() {
+ return offlineDataStore;
+ }
+
+
+}
diff --git a/src/PamController/pamBuoyGlobals.java b/src/PamController/pamBuoyGlobals.java
index dee3a1db..9b0a436f 100644
--- a/src/PamController/pamBuoyGlobals.java
+++ b/src/PamController/pamBuoyGlobals.java
@@ -14,9 +14,9 @@ public class pamBuoyGlobals {
// public static Integer useGstreamer = 0;
// public static boolean useNetworkCont = false;
private static Integer networkControlPort = null;
-private static String multiportAddress;
+private static String multicastAddress;
// private static boolean useDSP = false;
-private static int muliportPort;
+private static int mulicastPort;
/**
* @return the networkControlPort
@@ -33,28 +33,28 @@ private static int muliportPort;
}
/**
- * Set parameters for mulitport configutation.
+ * Set parameters for mulitport configuration.
* @param mAddr
* @param mPort
*/
public static void setMultiportConfig(String mAddr, int mPort) {
- multiportAddress = mAddr;
- muliportPort = mPort;
+ multicastAddress = mAddr;
+ mulicastPort = mPort;
}
/**
* @return the multiportAddress
*/
- public static String getMultiportAddress() {
- return multiportAddress;
+ public static String getMulticastAddress() {
+ return multicastAddress;
}
/**
* @return the muliportPort
*/
- public static int getMuliportPort() {
- return muliportPort;
+ public static int getMulticastPort() {
+ return mulicastPort;
}
diff --git a/src/PamController/settings/output/xml/PamguardXMLWriter.java b/src/PamController/settings/output/xml/PamguardXMLWriter.java
index 1e9aae0f..6b42c4e7 100644
--- a/src/PamController/settings/output/xml/PamguardXMLWriter.java
+++ b/src/PamController/settings/output/xml/PamguardXMLWriter.java
@@ -662,7 +662,7 @@ public class PamguardXMLWriter implements PamSettings {
}
}
catch (Exception e) {
- System.out.println(e.getMessage());
+ System.out.println("Error in PamguardXMLWriter.writeArray: " + e.getMessage());
}
return null;
}
@@ -704,7 +704,7 @@ public class PamguardXMLWriter implements PamSettings {
return e;
}
catch (Exception e) {
- System.out.println(e.getMessage());
+ System.out.println("Error in PamguardXMLWriter.writeObjectArray: " + e.getMessage());
}
return null;
}
@@ -729,7 +729,7 @@ public class PamguardXMLWriter implements PamSettings {
// el.appendChild(e);
}
catch (Exception e) {
- System.out.println(e.getMessage());
+ System.out.println("Error in PamguardXMLWriter.writePrimativeArray: " + e.getMessage());
}
return null;
}
diff --git a/src/PamModel/PamModel.java b/src/PamModel/PamModel.java
index 38539cfe..4ce627dc 100644
--- a/src/PamModel/PamModel.java
+++ b/src/PamModel/PamModel.java
@@ -460,8 +460,6 @@ final public class PamModel implements PamModelInterface, PamSettings {
mi.setModulesMenuGroup(utilitiesGroup);
mi.setMaxNumber(1);
-
-
/*
* ************* End Utilities Group *******************
*/
diff --git a/src/PamUtils/FileFunctions.java b/src/PamUtils/FileFunctions.java
index 6d3d1319..739a7a92 100644
--- a/src/PamUtils/FileFunctions.java
+++ b/src/PamUtils/FileFunctions.java
@@ -108,7 +108,7 @@ public class FileFunctions {
return attrib;
}
catch (Error e) {
- System.out.println(e.getMessage());
+ System.out.println("Error in FileFunctions.getAttributes: " + e.getMessage());
return 0;
}
}
diff --git a/src/PamUtils/PamCalendar.java b/src/PamUtils/PamCalendar.java
index 3ce2eebc..d41aa018 100644
--- a/src/PamUtils/PamCalendar.java
+++ b/src/PamUtils/PamCalendar.java
@@ -46,6 +46,8 @@ public class PamCalendar {
public static TimeZone defaultTimeZone = TimeZone.getTimeZone("UTC");
+
+ private static TimeZone localTimeZone = TimeZone.getDefault();
public static final long millisPerDay = 1000L*24L*3600L;
@@ -174,8 +176,9 @@ public class PamCalendar {
}
public static TimeZone getDisplayTimeZone(boolean useLocal) {
- return TimeZone.getTimeZone("UTC");
- // return useLocal ? CalendarControl.getInstance().getChosenTimeZone() : defaultTimeZone;
+// return TimeZone.getTimeZone("UTC");
+// return useLocal ? CalendarControl.getInstance().getChosenTimeZone() : defaultTimeZone;
+ return useLocal ? localTimeZone : defaultTimeZone;
}
public static String formatDateTime(Date date) {
diff --git a/src/PamUtils/worker/filelist/FileListWorker.java b/src/PamUtils/worker/filelist/FileListWorker.java
index 56c52caa..e71a5f62 100644
--- a/src/PamUtils/worker/filelist/FileListWorker.java
+++ b/src/PamUtils/worker/filelist/FileListWorker.java
@@ -8,6 +8,7 @@ import PamUtils.PamFileFilter;
import PamUtils.worker.PamWorkProgressMessage;
import PamUtils.worker.PamWorkWrapper;
import PamUtils.worker.PamWorker;
+import PamguardMVC.debug.Debug;
/**
* Class to list files in one or more directories.
@@ -64,6 +65,9 @@ public abstract class FileListWorker implements PamWorkWrapper>>>>>>>Starting file search in " + rootList[i]);
+ }
if (noChange(rootList, subFolders, useOldIfPossible)) {
SwingUtilities.invokeLater(new Runnable() {
@Override
@@ -162,6 +166,7 @@ public abstract class FileListWorker implements PamWorkWrapper> pamWorker, FileListData newFileList, File folder) {
newFileList.addFolder();
pamWorker.update(new PamWorkProgressMessage(-1, "Searching folder " + folder.getAbsolutePath()));
+ Debug.out.println(">>>> Searching for files in abs path " + folder.getAbsolutePath());
// System.out.println(folder.getAbsolutePath());
File[] moreFiles = folder.listFiles(fileFilter);
if (moreFiles == null) {
@@ -175,6 +180,7 @@ public abstract class FileListWorker implements PamWorkWrapper {
private SwingTableColumnWidths columnWidths;
+ /**
+ * Most work will run off a copy of the data.
+ * Makes it easier to include data selectors, etc.
+ */
+ private ArrayList dataUnitCopy;
+
+ private Object copySynch = new Object();
+
public DataBlockTableView(PamDataBlock pamDataBlock, String displayName) {
this.pamDataBlock = pamDataBlock;
this.displayName = displayName;
@@ -97,6 +106,13 @@ public abstract class DataBlockTableView {
}
}
+ /**
+ * Call the table data changed function to update table values.
+ */
+ public void fireTableDataChanged() {
+ blockTableModel.fireTableDataChanged();
+ }
+
public JComponent getComponent() {
return tablePanel;
}
@@ -141,10 +157,22 @@ public abstract class DataBlockTableView {
String tip = null;
java.awt.Point p = e.getPoint();
int rowIndex = rowAtPoint(p);
+// int column = columnAtPoint(p);
+// System.out.println("Get tooltip for row " + rowIndex + " column" + column);
+ if (rowIndex < 0) {
+ return null;
+ }
int colIndex = columnAtPoint(p);
int realColumnIndex = convertColumnIndexToModel(colIndex);
- T dataUnit = getDataUnit(rowIndex);
- return DataBlockTableView.this.getToolTipText(dataUnit, realColumnIndex);
+ try {
+ T dataUnit = getDataUnit(rowIndex);
+ return DataBlockTableView.this.getToolTipText(dataUnit, realColumnIndex);
+ }
+ catch (Exception ex) {
+ // I once got an index out of bounds here, or it might have been
+ // a concurrent modification exception ? Either way return null;
+ return null;
+ }
}
}
@@ -201,21 +229,31 @@ public abstract class DataBlockTableView {
* @return data unit for the table row.
*/
private final T getDataUnit(int tableRow) {
- synchronized (pamDataBlock.getSynchLock()) {
+ synchronized (copySynch) {
int rowIndex = getDataIndexForRow(tableRow);
if (rowIndex < 0) return null;
- return pamDataBlock.getDataUnit(rowIndex, PamDataBlock.REFERENCE_CURRENT);
+ if (dataUnitCopy == null) {
+ return null;
+ }
+ return dataUnitCopy.get(tableRow);
+// return pamDataBlock.getDataUnit(rowIndex, PamDataBlock.REFERENCE_CURRENT);
}
}
/**
* Get the number of rows in the table - default behaviour is the
- * number of rows in the datablock, but this may be overridded if
+ * number of rows in the datablock, but this may be overridden if
* data are being selected in a different way.
- * @return numer of table rows to show.
+ * @return number of table rows to show.
*/
public int getRowCount() {
- return pamDataBlock.getUnitsCount();
+ if (dataUnitCopy == null) {
+ updatePamData();
+ if (dataUnitCopy == null) {
+ return 0;
+ }
+ }
+ return dataUnitCopy.size();
}
/**
@@ -227,7 +265,10 @@ public abstract class DataBlockTableView {
* @return
*/
public int getDataIndexForRow(int tableRow) {
- int nRow = pamDataBlock.getUnitsCount();
+ if (dataUnitCopy == null) {
+ return tableRow;
+ }
+ int nRow = dataUnitCopy.size();
if (!isViewer) {
tableRow = nRow-tableRow-1;
}
@@ -244,12 +285,14 @@ public abstract class DataBlockTableView {
@Override
public void addData(PamObservable o, PamDataUnit arg) {
- blockTableModel.fireTableDataChanged();
+ DataBlockTableView.this.updatePamData();
+// blockTableModel.fireTableDataChanged();
}
@Override
public void updateData(PamObservable observable, PamDataUnit pamDataUnit) {
- blockTableModel.fireTableDataChanged();
+ DataBlockTableView.this.updatePamData();
+// blockTableModel.fireTableDataChanged();
}
@Override
@@ -258,6 +301,15 @@ public abstract class DataBlockTableView {
}
}
+
+ private void updatePamData() {
+ synchronized (copySynch) {
+ dataUnitCopy = pamDataBlock.getDataCopy();
+ }
+ blockTableModel.fireTableDataChanged();
+ }
+
+
private class MouseAction extends MouseAdapter {
/* (non-Javadoc)
@@ -304,7 +356,11 @@ public abstract class DataBlockTableView {
* @return Array of multiple rows selected.
*/
public T[] getMultipleSelectedRows() {
- synchronized(pamDataBlock.getSynchLock()) {
+ if (dataUnitCopy == null) {
+ return null;
+ }
+// synchronized(pamDataBlock.getSynchLock()) { // synch not needed with data copy.
+ synchronized (copySynch) {
int[] selRows = testTable.getSelectedRows();
if (selRows == null) {
return null;
@@ -337,12 +393,14 @@ public abstract class DataBlockTableView {
@Override
public void scrollValueChanged(AbstractPamScroller abstractPamScroller) {
- blockTableModel.fireTableDataChanged();
+// blockTableModel.fireTableDataChanged();
+ updatePamData();
}
@Override
public void scrollRangeChanged(AbstractPamScroller pamScroller) {
- blockTableModel.fireTableDataChanged();
+// blockTableModel.fireTableDataChanged();
+ updatePamData();
}
}
diff --git a/src/PamguardMVC/PamDataBlock.java b/src/PamguardMVC/PamDataBlock.java
index 2b987ffa..2ad1d09e 100644
--- a/src/PamguardMVC/PamDataBlock.java
+++ b/src/PamguardMVC/PamDataBlock.java
@@ -2078,6 +2078,9 @@ public class PamDataBlock extends PamObservable {
/*
* Doesnt notify it's own parent to avoid an infinite loop
*/
+ if (Float.isNaN(sampleRate)) {
+ System.out.println("NaN sample rate being set in " + getLongDataName());
+ }
if (notify) {
for (int i = 0; i < countObservers(); i++) {
if (getPamObserver(i).getObserverObject() != parentProcess) {
diff --git a/src/PamguardMVC/uid/binaryUIDFunctions.java b/src/PamguardMVC/uid/binaryUIDFunctions.java
index f8959935..310f3b23 100644
--- a/src/PamguardMVC/uid/binaryUIDFunctions.java
+++ b/src/PamguardMVC/uid/binaryUIDFunctions.java
@@ -46,6 +46,7 @@ import javax.xml.xpath.XPathConstants;
import javax.xml.xpath.XPathExpression;
import javax.xml.xpath.XPathFactory;
+import org.apache.commons.io.comparator.NameFileComparator;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
@@ -391,7 +392,7 @@ public class binaryUIDFunctions {
// System.out.println("Warning - no " + filePrefix + " binary files found in " + binStore.getBinaryStoreSettings().getStoreLocation());
return maxUID;
}
- Collections.sort(binFiles);
+ Collections.sort(binFiles, NameFileComparator.NAME_COMPARATOR);
// loop through the binary files from the last one to the first, and stop as
// soon as we find
diff --git a/src/RightWhaleEdgeDetector/RWEProcess.java b/src/RightWhaleEdgeDetector/RWEProcess.java
index 07a0584c..ccc766bb 100644
--- a/src/RightWhaleEdgeDetector/RWEProcess.java
+++ b/src/RightWhaleEdgeDetector/RWEProcess.java
@@ -177,6 +177,7 @@ public class RWEProcess extends PamProcess {
minSoundType = rweControl.rweParameters.minSoundType;
classifier.setSoundData(getSampleRate(), sourceDataBlock.getFftLength(),
sourceDataBlock.getFftHop());
+// System.out.println("Create right whale channel process " + iChannel);
}
diff --git a/src/alarm/AlarmProcess.java b/src/alarm/AlarmProcess.java
index c6194ced..a8e3ee36 100644
--- a/src/alarm/AlarmProcess.java
+++ b/src/alarm/AlarmProcess.java
@@ -225,6 +225,11 @@ public class AlarmProcess extends PamProcess {
}
+ @Override
+ public void updateData(PamObservable o, PamDataUnit arg) {
+ newData(o, arg);
+ }
+
public boolean setupAlarm() {
dataSource = PamController.getInstance().getDataBlock(PamDataUnit.class, alarmControl.alarmParameters.dataSourceName);
if (dataSource == null) {
diff --git a/src/alarm/actions/sound/PlaySound.java b/src/alarm/actions/sound/PlaySound.java
index 0d3a13fe..e02ac14f 100644
--- a/src/alarm/actions/sound/PlaySound.java
+++ b/src/alarm/actions/sound/PlaySound.java
@@ -94,7 +94,7 @@ public class PlaySound extends AlarmAction implements PamSettings {
double secs = fl/sr*1000;
}
catch (Exception e) {
- System.out.println(e.getMessage());
+ System.out.println("Error in PlaySound.playSound:" + e.getMessage());
return false;
}
return true;
@@ -114,10 +114,10 @@ public class PlaySound extends AlarmAction implements PamSettings {
System.out.println(f.getMessage());
return currentClip = null;
} catch (IOException e) {
- System.out.println(e.getMessage());
+ System.out.println("IOError in PlaySound.prepareClip: " + e.getMessage());
return currentClip = null;
} catch (LineUnavailableException e) {
- System.out.println(e.getMessage());
+ System.out.println("LineUnavailableException in PlaySound.prepareClip: " + e.getMessage());
return currentClip = null;
}
currentFile = soundFile;
diff --git a/src/backupmanager/stream/BackupStream.java b/src/backupmanager/stream/BackupStream.java
index ae4f87e4..bc0e3d14 100644
--- a/src/backupmanager/stream/BackupStream.java
+++ b/src/backupmanager/stream/BackupStream.java
@@ -118,7 +118,7 @@ public abstract class BackupStream implements PamSettings, BackupFunction {
try {
ok = doAction(backupManager, action, item);
} catch (BackupException e) {
- System.out.println(e.getMessage());
+ System.out.println("Error in BackupStream.runAction:" + e.getMessage());
backupManager.updateProgress(new BackupProgress(this, action, STATE.PROBLEM, toDoList.size(),
iDone, "Error " + e.getMessage()));
ok = false; //otherwise, OK stays true! This will leave a true null entry in the database, so file will be selected next time around
diff --git a/src/binaryFileStorage/BinaryDataSource.java b/src/binaryFileStorage/BinaryDataSource.java
index ccd07440..bb2888ba 100644
--- a/src/binaryFileStorage/BinaryDataSource.java
+++ b/src/binaryFileStorage/BinaryDataSource.java
@@ -41,6 +41,8 @@ public abstract class BinaryDataSource {
private boolean saveUpdates = false;
private BackgroundBinaryWriter backgroundBinaryWriter;
+
+ public static final Object packSynchObject = new Object();
/**
* Create a binary data source. These are used both to store data in binary
@@ -224,11 +226,13 @@ public abstract class BinaryDataSource {
* Then pack the data
*/
BinaryObjectData data;
- if (pamDataUnit instanceof BackgroundDataUnit) {
- data = getBackgroundBinaryWriter().packBackgroundData((BackgroundDataUnit) pamDataUnit);
- }
- else {
- data = getPackedData(pamDataUnit);
+ synchronized (packSynchObject) {
+ if (pamDataUnit instanceof BackgroundDataUnit) {
+ data = getBackgroundBinaryWriter().packBackgroundData((BackgroundDataUnit) pamDataUnit);
+ }
+ else {
+ data = getPackedData(pamDataUnit);
+ }
}
/*
* Then put it back to how it was a moment ago.
diff --git a/src/binaryFileStorage/BinaryOutputStream.java b/src/binaryFileStorage/BinaryOutputStream.java
index 6314c88e..1e474686 100644
--- a/src/binaryFileStorage/BinaryOutputStream.java
+++ b/src/binaryFileStorage/BinaryOutputStream.java
@@ -117,6 +117,8 @@ public class BinaryOutputStream {
File outputFile = new File(mainFileName);
boolean open = openPGDFFile(outputFile);
+
+// System.out.println("Open outout file " + outputFile.getAbsolutePath());
if (open) {
addToDataMap(outputFile);
@@ -260,6 +262,7 @@ public class BinaryOutputStream {
public synchronized boolean closeFile() {
boolean ok = true;
+// System.out.println("Close output file " + mainFileName);
if (dataOutputStream != null) {
if (currentDataMapPoint != null) {
currentDataMapPoint.setBinaryFooter(footer);
diff --git a/src/binaryFileStorage/BinaryStore.java b/src/binaryFileStorage/BinaryStore.java
index c2007cb2..f3965911 100644
--- a/src/binaryFileStorage/BinaryStore.java
+++ b/src/binaryFileStorage/BinaryStore.java
@@ -36,6 +36,8 @@ import dataGram.DatagramManager;
import dataMap.OfflineDataMap;
import dataMap.OfflineDataMapPoint;
import PamController.AWTScheduler;
+import PamController.DataInputStore;
+import PamController.DataOutputStore;
import PamController.OfflineDataStore;
import PamController.PamControlledUnit;
import PamController.PamControlledUnitGUI;
@@ -49,6 +51,7 @@ import PamController.PamSettingsGroup;
import PamController.PamSettingsSource;
import PamController.StorageOptions;
import PamController.StorageParameters;
+import PamController.fileprocessing.StoreStatus;
import PamController.status.ModuleStatus;
import PamController.status.QuickRemedialAction;
import PamModel.SMRUEnable;
@@ -89,7 +92,7 @@ import binaryFileStorage.layoutFX.BinaryStoreGUIFX;
*
*/
public class BinaryStore extends PamControlledUnit implements PamSettings,
-PamSettingsSource, OfflineDataStore {
+PamSettingsSource, DataOutputStore {
public static final String fileType = "pgdf";
@@ -1113,6 +1116,8 @@ PamSettingsSource, OfflineDataStore {
String lastFailedStream = null;
+ private BinaryStoreStatusFuncs binaryStoreStatusFuncs;
+
public boolean removeMapPoint(File aFile, ArrayList streams) {
BinaryHeaderAndFooter bhf = getFileHeaderAndFooter(aFile);
if (bhf == null || bhf.binaryHeader == null) {
@@ -1354,6 +1359,26 @@ PamSettingsSource, OfflineDataStore {
}
return newFile;
}
+
+ /**
+ * Find the noise file to match a given data file.
+ * @param dataFile data file.
+ * @param checkExists check the file exists and if it doens't return null
+ * @return index file to go with the data file.
+ */
+ public File findNoiseFile(File dataFile, boolean checkExists) {
+// String filePath = dataFile.getAbsolutePath();
+// // check that the last 4 characters are "pgdf"
+// int pathLen = filePath.length();
+// String newPath = filePath.substring(0, pathLen-4) + indexFileType;
+ File newFile = swapFileType(dataFile, noiseFileType);
+ if (checkExists) {
+ if (newFile.exists() == false) {
+ return null;
+ }
+ }
+ return newFile;
+ }
/**
* Create an index file (pgdx) name from a data file (pgdf) file name
@@ -1488,7 +1513,7 @@ PamSettingsSource, OfflineDataStore {
* @param folder folder to search
* @param filter file filter
*/
- private void listDataFiles(ArrayList fileList, File folder, PamFileFilter filter) {
+ public void listDataFiles(ArrayList fileList, File folder, PamFileFilter filter) {
File[] newFiles = folder.listFiles(filter);
if (newFiles == null) {
return;
@@ -1684,7 +1709,7 @@ PamSettingsSource, OfflineDataStore {
* @param binaryObjectData
* @param dataSink
*/
- private void unpackAnnotationData(int fileVersion, PamDataUnit createdUnit, BinaryObjectData binaryObjectData, BinaryDataSink dataSink) {
+ protected void unpackAnnotationData(int fileVersion, PamDataUnit createdUnit, BinaryObjectData binaryObjectData, BinaryDataSink dataSink) {
//System.out.println("Hello annotation " + binaryObjectData.getAnnotationDataLength());
if (binaryObjectData.getAnnotationDataLength() == 0) {
@@ -2211,7 +2236,7 @@ PamSettingsSource, OfflineDataStore {
- private boolean reportError(String string) {
+ boolean reportError(String string) {
System.out.println(string);
return false;
}
@@ -2379,6 +2404,7 @@ PamSettingsSource, OfflineDataStore {
}
else {
reportError("No valid header in file " + file.getAbsolutePath());
+ inputStream.closeFile();
return null;
}
@@ -2509,4 +2535,17 @@ PamSettingsSource, OfflineDataStore {
return binaryStoreSettings.getNoiseStoreType();
}
+ @Override
+ public StoreStatus getStoreStatus(boolean getDetail) {
+ if (binaryStoreStatusFuncs == null) {
+ binaryStoreStatusFuncs = new BinaryStoreStatusFuncs(this);
+ }
+ return binaryStoreStatusFuncs.getStoreStatus(getDetail);
+ }
+ @Override
+ public boolean deleteDataFrom(long timeMillis) {
+ BinaryStoreDeleter storeDeleter = new BinaryStoreDeleter(this);
+ return storeDeleter.deleteDataFrom(timeMillis);
+ }
+
}
diff --git a/src/binaryFileStorage/BinaryStoreDeleter.java b/src/binaryFileStorage/BinaryStoreDeleter.java
new file mode 100644
index 00000000..13401e04
--- /dev/null
+++ b/src/binaryFileStorage/BinaryStoreDeleter.java
@@ -0,0 +1,369 @@
+package binaryFileStorage;
+
+import java.io.BufferedInputStream;
+import java.io.DataInputStream;
+import java.io.File;
+import java.io.FileFilter;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+import org.apache.commons.io.comparator.NameFileComparator;
+
+import PamUtils.PamCalendar;
+import PamUtils.PamFileFilter;
+import PamguardMVC.DataUnitBaseData;
+import PamguardMVC.PamDataBlock;
+import PamguardMVC.PamDataUnit;
+
+public class BinaryStoreDeleter {
+
+ private static final int FILE_DELETE_ERROR = 1;
+ private static final int FILE_TOO_EARLY = 2;
+ private static final int FILE_DELETED = 3;
+ private static final int FILE_PARTIAL_DELETE = 4;
+
+
+ private BinaryStore binaryStore;
+
+ private FileFilter directoryFilter;
+
+ private BinaryStoreStatusFuncs binaryStoreStatusFuncs;
+
+ public BinaryStoreDeleter(BinaryStore binaryStore) {
+ this.binaryStore = binaryStore;
+ directoryFilter = new DirectoryFilter();
+ binaryStoreStatusFuncs = new BinaryStoreStatusFuncs(binaryStore);
+ }
+
+ public boolean deleteDataFrom(long timeMillis) {
+ if (timeMillis == 0) {
+ return deleteEverything();
+ }
+ else {
+ return deleteFrom(timeMillis);
+ }
+ }
+
+ private class DirectoryFilter implements java.io.FileFilter {
+
+ @Override
+ public boolean accept(File pathname) {
+ return pathname.isDirectory();
+ }
+
+ }
+
+ private boolean deleteEverything() {
+ ArrayList fileList = new ArrayList();
+ String root = binaryStore.binaryStoreSettings.getStoreLocation();
+ if (root == null) {
+ return false;
+ }
+ File rootFolder = new File(root);
+ PamFileFilter binaryDataFilter = new PamFileFilter("Binary Data Files", BinaryStore.fileType);
+ binaryDataFilter.addFileType(BinaryStore.indexFileType);
+ binaryDataFilter.addFileType(BinaryStore.noiseFileType);
+ binaryDataFilter.setAcceptFolders(true);
+
+ binaryStore.listDataFiles(fileList, rootFolder, binaryDataFilter);
+ int errors = 0;
+ for (File aFile : fileList) {
+ try {
+ aFile.delete();
+ }
+ catch (Exception e) {
+ errors++;
+ }
+ }
+ deleteEmptyFolders();
+ return errors == 0;
+ }
+
+ private boolean deleteFrom(long timeMillis) {
+ /*
+ * need to go through the data one stream at a time so that
+ * we can pick files off from the end of the list.
+ */
+ ArrayList streams = BinaryStore.getStreamingDataBlocks(true);
+ int errors = 0;
+ for (PamDataBlock aBlock : streams) {
+ boolean ok = deleteFrom(aBlock, timeMillis);
+ if (!ok) {
+ errors++;
+ }
+ }
+
+ deleteEmptyFolders();
+ return false;
+ }
+
+ private boolean deleteFrom(PamDataBlock aBlock, long timeMillis) {
+ System.out.printf("Deleting binary data for %s from %s\n", aBlock.getDataName(), PamCalendar.formatDBDateTime(timeMillis));
+ BinaryDataSource dataSource = aBlock.getBinaryDataSource();
+ if (dataSource == null) {
+ return true; // don't see how this can happen.
+ }
+ // first deal with pgdf and pgdx files, then noise.
+ String filePrefix = dataSource.createFilenamePrefix();
+ List binFiles = binaryStore.listAllFilesWithPrefix(filePrefix);
+ if (binFiles == null || binFiles.isEmpty()) {
+ return true; // nothing to delete.
+ }
+ Collections.sort(binFiles, NameFileComparator.NAME_COMPARATOR);
+ for (int i = binFiles.size()-1; i >= 0; i--) {
+ int ans = deleteFileFrom(aBlock, binFiles.get(i), timeMillis);
+ if (ans == FILE_TOO_EARLY) {
+ break;
+ }
+ }
+
+ return true;
+ }
+
+ /**
+ * Delete a specific file from a specific time. If the start of the file
+ * is after timeMillis, delete the entire file, otherwise it will have
+ * to be a partial delete.
+ * @param aBlock
+ * @param file
+ * @param timeMillis
+ * @return
+ */
+ private int deleteFileFrom(PamDataBlock aBlock, File dataFile, long timeMillis) {
+ File indexFile = binaryStore.findIndexFile(dataFile, true);
+ if (indexFile == null) {
+ indexFile = dataFile;
+ }
+ File noiseFile = binaryStore.findNoiseFile(dataFile, true);
+ // get the header.
+ boolean headOk = false;
+ BinaryHeader binaryHead = new BinaryHeader();
+ try {
+ FileInputStream fis = new FileInputStream(indexFile);
+ DataInputStream dis = new DataInputStream(new BufferedInputStream(fis));
+ headOk = binaryHead.readHeader(dis);
+ fis.close();
+ }
+ catch (IOException e) {
+ headOk = false;
+ }
+ if (headOk == false || binaryHead.getDataDate() >= timeMillis) {
+ boolean deleteOk = deleteFileSet(dataFile);
+ return deleteOk ? FILE_DELETED : FILE_DELETE_ERROR;
+ }
+ /**
+ * Now need to see if the file is earlier than we want, in which case we return
+ * immediately and won't look at any more files.
+ */
+ BinaryFooter fileEnd = binaryStoreStatusFuncs.findLastData(dataFile);
+ if (fileEnd == null) {
+ // the file has no footer and no data, so must be corrupt, so delete it.
+ boolean deleteOk = deleteFileSet(dataFile);
+ return deleteOk ? FILE_DELETED : FILE_DELETE_ERROR;
+ }
+ if (fileEnd.getDataDate() <= timeMillis) {
+ /*
+ * this file is earlier than our delete time, so we don't want to delete it
+ * and need to send a message saying not to delete anything else either.
+ */
+ return FILE_TOO_EARLY;
+ }
+ /**
+ * If we land here, it looks like we're in the realm of needing to partially delete
+ * a file / set of data and noise files. What a pain ! Will need to do
+ * the deleting and update the index file. f** knows what to do about a
+ * serialized datamap.
+ */
+ partialCopyFile(aBlock, dataFile, timeMillis);
+ if (indexFile != null) {
+ partialCopyFile(aBlock, indexFile, timeMillis);
+ }
+ if (noiseFile != null) {
+ partialCopyFile(aBlock, noiseFile, timeMillis);
+ }
+
+ return FILE_PARTIAL_DELETE;
+ }
+
+ private boolean partialCopyFile(PamDataBlock aBlock, File dataFile, long timeMillis) {
+ System.out.printf("Partial delete of file %s from %s\n", dataFile.getAbsoluteFile(), PamCalendar.formatDBDateTime(timeMillis));
+ try {
+ BinaryInputStream inputStream = new BinaryInputStream(binaryStore, aBlock);
+ if (inputStream.openFile(dataFile) == false) {
+ return false;
+ }
+
+ BinaryDataSource dataSource = aBlock.getBinaryDataSource();
+
+ File tempFile = new File(dataFile.getAbsolutePath() + ".tmp");
+ BinaryOutputStream outputStream = new BinaryOutputStream(binaryStore, aBlock);
+ dataSource.setBinaryStorageStream(outputStream);
+
+ BinaryObjectData binaryObjectData;
+ BinaryHeader bh = inputStream.readHeader();
+ if (bh==null) {
+ return false;
+ }
+ outputStream.writeHeader(bh.getDataDate(), bh.getAnalysisDate());
+ ModuleHeader mh = null;
+
+ BinaryFooter bf = null;
+ int inputFormat = bh.getHeaderFormat();
+ while ((binaryObjectData = inputStream.readNextObject(inputFormat)) != null) {
+
+ switch (binaryObjectData.getObjectType()) {
+ case BinaryTypes.FILE_FOOTER:
+ // this is unlikely to happen, since we'll probably already have found an index file.
+ bf = new BinaryFooter();
+ bf.readFooterData(binaryObjectData.getDataInputStream(), inputFormat);
+ bf.setDataDate(timeMillis);
+ outputStream.writeFileFooter(bf);
+ break;
+ case BinaryTypes.MODULE_HEADER:
+ mh = dataSource.sinkModuleHeader(binaryObjectData, bh);
+ outputStream.writeModuleHeader();
+ break;
+ case BinaryTypes.MODULE_FOOTER:
+ ModuleFooter mf = dataSource.sinkModuleFooter(binaryObjectData, bh, mh);
+ outputStream.writeModuleFooter();
+ break;
+ case BinaryTypes.DATAGRAM:
+// dataSource.
+ break;
+ default: // should be data.
+ DataUnitBaseData baseData = binaryObjectData.getDataUnitBaseData();
+ if (baseData == null) {
+ continue;
+ }
+ if (baseData.getTimeMilliseconds() > timeMillis) {
+ continue;
+ }
+ /*
+ * otherwise we need to store this data unit. I think we can just copy in the
+ * existing binary data to the new file non ? Might mess the datagram slightly,
+ * but that is only in the index file and can sort itself out.
+ * better to make a data unit and then rewrite it I think.
+ */
+ PamDataUnit dataUnit = dataSource.sinkData(binaryObjectData, bh, inputFormat);
+ if (dataUnit != null) {
+ dataUnit.getBasicData().mergeBaseData(binaryObjectData.getDataUnitBaseData());
+ binaryStore.unpackAnnotationData(bh.getHeaderFormat(), dataUnit, binaryObjectData, null);
+ dataSource.saveData(dataUnit);
+ }
+
+ }
+ }
+
+ outputStream.closeFile();
+ inputStream.closeFile();
+
+ /*
+ * Now file final stage - copy the temp file in place of the
+ * original file.
+ */
+ boolean deletedOld = false;
+ try {
+ deletedOld = dataFile.delete();
+ }
+ catch (SecurityException e) {
+ System.out.println("Error deleting old pgdf file: " + dataFile.getAbsolutePath());
+ e.printStackTrace();
+ }
+
+ boolean renamedNew = false;
+ try {
+ renamedNew = tempFile.renameTo(dataFile);
+ }
+ catch (SecurityException e) {
+ System.out.println("Error renaming new pgdf file: " + tempFile.getAbsolutePath() +
+ " to " + dataFile.getAbsolutePath());
+ e.printStackTrace();
+ }
+ if (renamedNew == false) {
+ if (deletedOld == false) {
+ binaryStore.reportError("Unable to delete " + dataFile.getAbsolutePath());
+ }
+ return binaryStore.reportError(String.format("Unable to rename %s to %s",
+ tempFile.getAbsolutePath(), dataFile.getAbsolutePath()));
+ }
+
+ return true;
+
+ }
+ catch (Exception ex) {
+ return false;
+ }
+
+ }
+
+ /**
+ * Delete a set of files, including main data file, index file and noise file.
+ * @param dataFile
+ * @return
+ */
+ private boolean deleteFileSet(File dataFile) {
+ System.out.printf("Deleting full file set for %s\n", dataFile.getAbsoluteFile());
+ boolean deleteOk = true;
+ try {
+ File indexFile = binaryStore.findIndexFile(dataFile, true);
+ File noiseFile = binaryStore.findNoiseFile(dataFile, true);
+ deleteOk &= dataFile.delete();
+ if (indexFile != null) {
+ deleteOk &= indexFile.delete();
+ }
+ if (noiseFile != null) {
+ deleteOk &= noiseFile.delete();
+ }
+ }
+ catch (Exception e) {
+ deleteOk = false;
+ }
+
+ System.out.printf("Deleting full file set %s for %s\n", deleteOk?"OK":"Error", dataFile.getAbsoluteFile());
+
+ return deleteOk;
+
+ }
+
+ private void deleteEmptyFolders() {
+ String root = binaryStore.binaryStoreSettings.getStoreLocation();
+ if (root == null) {
+ return;
+ }
+ /**
+ * Iterate through the root folder first and then call a
+ * recursive function to delete sub folders. this will stop the
+ * root folder from being deleted, but sub folders will get deleted if
+ * they have no files (of any type) in them.
+ */
+ File rootFolder = new File(root);
+ File[] subFolders = rootFolder.listFiles(directoryFilter);
+ if (subFolders == null) {
+ return;
+ }
+ for (int i = 0; i < subFolders.length; i++) {
+ deleteEmptyFolders(subFolders[i]);
+ }
+ }
+
+ private void deleteEmptyFolders(File file) {
+ File[] subFolders = file.listFiles(directoryFilter);
+ for (int i = 0; i < subFolders.length; i++) {
+ deleteEmptyFolders(subFolders[i]);
+ }
+ // see if there is anything at all in this folder
+ File[] remaining = file.listFiles();
+ if (remaining.length == 0) {
+ try {
+ file.delete();
+ }
+ catch (Exception e) {
+ System.out.printf("Binary folder %s cannot be deleted: %s\n", file.getAbsolutePath(), e.getMessage());
+ }
+ }
+ }
+
+}
diff --git a/src/binaryFileStorage/BinaryStoreProcess.java b/src/binaryFileStorage/BinaryStoreProcess.java
index 7d653fbc..c3a728b4 100644
--- a/src/binaryFileStorage/BinaryStoreProcess.java
+++ b/src/binaryFileStorage/BinaryStoreProcess.java
@@ -46,7 +46,7 @@ public class BinaryStoreProcess extends PamProcess {
startTime = PamCalendar.getTimeInMillis();
long round = binaryStore.binaryStoreSettings.fileSeconds * 1000;
nextFileTime = (startTime/round) * round + round;
- System.out.println("Next file start at " + PamCalendar.formatDateTime(nextFileTime));
+// System.out.println("Next file start at " + PamCalendar.formatDateTime(nextFileTime));
timer = new Timer();
timer.schedule(new FileTimerTask(), 1000, 1000);
diff --git a/src/binaryFileStorage/BinaryStoreStatus.java b/src/binaryFileStorage/BinaryStoreStatus.java
new file mode 100644
index 00000000..11f20d3a
--- /dev/null
+++ b/src/binaryFileStorage/BinaryStoreStatus.java
@@ -0,0 +1,169 @@
+package binaryFileStorage;
+
+import PamController.fileprocessing.StoreStatus;
+
+public class BinaryStoreStatus extends StoreStatus {
+
+ private BinaryStore binaryStore;
+
+ private BinaryHeader firstHeader;
+
+ private BinaryFooter lastFooter;
+
+ private BinaryFooter lastData;
+
+ public BinaryStoreStatus(BinaryStore binaryStore) {
+ super(binaryStore);
+ this.binaryStore = binaryStore;
+ }
+
+ public BinaryStoreStatus(BinaryStore binaryStore, BinaryHeader firstHead, BinaryFooter lastFoot,
+ BinaryFooter lastData) {
+ super(binaryStore);
+ this.binaryStore = binaryStore;
+ this.firstHeader = firstHead;
+ this.lastFooter = lastFoot;
+ this.lastData = lastData;
+ }
+
+
+ @Override
+ public Long getFirstDataTime() {
+ if (firstHeader != null) {
+ return firstHeader.getDataDate();
+ }
+ return null;
+ }
+
+ @Override
+ public Long getLastDataTime() {
+ if (lastData != null) {
+ return lastData.getDataDate();
+ }
+ if (lastFooter != null) {
+ return lastFooter.getDataDate();
+ }
+ return null;
+ }
+
+ /**
+ * @return the firstHeader
+ */
+ public BinaryHeader getFirstHeader() {
+ return firstHeader;
+ }
+
+ /**
+ * @param firstHeader the firstHeader to set
+ */
+ public void setFirstHeader(BinaryHeader firstHeader) {
+ this.firstHeader = firstHeader;
+ if (firstHeader != null) {
+ setFirstDataTime(firstHeader.getDataDate());
+ }
+ else {
+ setFirstDataTime(null);
+ }
+ }
+
+ /**
+ * @return the lastFooter
+ */
+ public BinaryFooter getLastFooter() {
+ return lastFooter;
+ }
+
+ /**
+ * @param lastFooter the lastFooter to set
+ */
+ public void setLastFooter(BinaryFooter lastFooter) {
+ this.lastFooter = lastFooter;
+ }
+
+ /**
+ * @return the lastData
+ */
+ public BinaryFooter getLastData() {
+ return lastData;
+ }
+
+ /**
+ * @param lastData the lastData to set
+ */
+ public void setLastData(BinaryFooter lastData) {
+ this.lastData = lastData;
+ if (lastData != null) {
+ setLastDataTime(lastData.getDataDate());
+ }
+ else {
+ setLastDataTime(null);
+ }
+ }
+
+ @Override
+ public long getFreeSpace() {
+ return getFreeSpace(binaryStore.getBinaryStoreSettings().getStoreLocation());
+ }
+
+ /**
+ * Looking overall for first header, last footers, etc.
+ * @param blockStatus
+ */
+ public void considerBlockStatus(BinaryStoreStatus blockStatus) {
+ if (blockStatus == null) {
+ return;
+ }
+ considerFirstHeader(blockStatus.firstHeader);
+ considerLastFooter(blockStatus.lastFooter);
+ considerLastData(blockStatus.lastData);
+ }
+
+ /**
+ * Take a footer for last data with the later date
+ * @param footer
+ */
+ private void considerLastData(BinaryFooter footer) {
+ if (footer == null || footer.getDataDate() == 0) {
+ return;
+ }
+ if (lastData == null || lastData.getDataDate() == 0) {
+ lastData = footer;
+ }
+ if (footer.getDataDate() > lastData.getDataDate()) {
+ lastData = footer;
+ }
+ }
+
+ /**
+ * Take a footer for last footer with the later date
+ * @param footer
+ */
+ private void considerLastFooter(BinaryFooter footer) {
+ if (footer == null || footer.getDataDate() == 0) {
+ return;
+ }
+ if (lastFooter == null || lastFooter.getDataDate() == 0) {
+ lastFooter = footer;
+ }
+ if (footer.getDataDate() > lastFooter.getDataDate()) {
+ lastFooter = footer;
+ }
+ }
+
+ /**
+ * Take a header for the first header with the earliest date.
+ * @param header
+ */
+ private void considerFirstHeader(BinaryHeader header) {
+ if (header == null || header.getDataDate() == 0) {
+ return;
+ }
+ if (this.firstHeader == null || firstHeader.getDataDate() == 0) {
+ this.firstHeader = header;
+ }
+ if (header.getDataDate() < firstHeader.getDataDate()) {
+ firstHeader = header;
+ }
+ }
+
+}
diff --git a/src/binaryFileStorage/BinaryStoreStatusFuncs.java b/src/binaryFileStorage/BinaryStoreStatusFuncs.java
new file mode 100644
index 00000000..c19447b2
--- /dev/null
+++ b/src/binaryFileStorage/BinaryStoreStatusFuncs.java
@@ -0,0 +1,287 @@
+package binaryFileStorage;
+
+import java.io.BufferedInputStream;
+import java.io.DataInputStream;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+import org.apache.commons.io.comparator.NameFileComparator;
+
+import PamController.fileprocessing.StoreStatus;
+import PamUtils.PamFileFilter;
+import PamguardMVC.DataUnitBaseData;
+import PamguardMVC.PamDataBlock;
+
+/**
+ * Set of functions used at restarts to determine the status of the binary store.
+ * @author dg50
+ *
+ */
+public class BinaryStoreStatusFuncs {
+
+ private BinaryStore binaryStore;
+
+ public BinaryStoreStatusFuncs(BinaryStore binaryStore) {
+ this.binaryStore = binaryStore;
+ }
+
+ public StoreStatus getStoreStatus(boolean getDetail) {
+ BinaryStoreStatus binStoreStatus = new BinaryStoreStatus(binaryStore);
+ binStoreStatus.setStoreStatus(checkStoreStatus());
+ if (getDetail && binStoreStatus.getStoreStatus() == StoreStatus.STATUS_HASDATA) {
+ binStoreStatus = getStoreDetail(binStoreStatus);
+ }
+ return binStoreStatus;
+ }
+
+ private BinaryStoreStatus getStoreDetail(BinaryStoreStatus binStoreStatus) {
+ // go through every stream and find it's first and last data times.
+ long lastTime = Long.MIN_VALUE;
+ long firstTime = Long.MAX_VALUE;
+ ArrayList streams = BinaryStore.getStreamingDataBlocks(true);
+ for (PamDataBlock aBlock : streams) {
+ BinaryDataSource dataSource = aBlock.getBinaryDataSource();
+ if (dataSource == null) {
+ continue;
+ }
+ BinaryStoreStatus blockStatus = getStreamStartEnd(dataSource);
+ if (blockStatus == null) {
+ continue;
+ }
+ binStoreStatus.considerBlockStatus(blockStatus);
+ }
+
+ return binStoreStatus;
+ }
+
+
+ private BinaryStoreStatus getStreamStartEnd(BinaryDataSource dataSource) {
+ String filePrefix = dataSource.createFilenamePrefix();
+ List binFiles = binaryStore.listAllFilesWithPrefix(filePrefix);
+ if (binFiles == null || binFiles.isEmpty()) {
+ return null;
+ }
+ Collections.sort(binFiles, NameFileComparator.NAME_COMPARATOR);
+ BinaryHeader firstHead = findFirstHeader(binFiles);
+ BinaryFooter lastFoot = findLastFooter(binFiles);
+ BinaryFooter lastData = findLastData(binFiles);
+ BinaryStoreStatus storeStatus = new BinaryStoreStatus(binaryStore, firstHead, lastFoot, lastData);
+ return storeStatus;
+ }
+
+ /**
+ * Get the last footer. This may be in the last file, but may not be if things
+ * crashed and the last file didn't get completed, i nwhich case it will be in
+ * the file before.
+ * @param binFiles
+ * @return
+ */
+ private BinaryFooter findLastFooter(List binFiles) {
+ for (int i = binFiles.size()-1; i>=0; i--) {
+ File aFile = binFiles.get(i);
+ /*
+ * if the last file was completed correctly, it will have an index file. If there isn't
+ * an index file it's very unlikely there will be a footer in the main file
+ */
+ File indexFile = binaryStore.findIndexFile(aFile, true);
+ if (indexFile == null) {
+ continue;
+ }
+ BinaryHeaderAndFooter headAndFoot = binaryStore.readHeaderAndFooter(indexFile);
+ if (headAndFoot != null && headAndFoot.binaryFooter != null) {
+ return headAndFoot.binaryFooter;
+ }
+ }
+ return null;
+ }
+
+ /**
+ * Get the last time of any data, whether it's from a header, footer, or actual data.
+ * @param binFiles
+ * @return
+ */
+ private BinaryFooter findLastData(List binFiles) {
+ for (int i = binFiles.size()-1; i>=0; i--) {
+ File aFile = binFiles.get(i);
+ BinaryFooter bf = findLastData(aFile);
+ if (bf != null) {
+ return bf;
+ }
+ }
+ return null;
+ }
+
+ /**
+ * Get the last data in a file. Hopefully this comes
+ * from the footer, but it might have to look at all data if
+ * the footer is absent or the index file missing.
+ * @param aFile
+ * @return
+ */
+ public BinaryFooter findLastData(File aFile) {
+ Long lastUID = null;
+ Long lastTime = null;
+ Long firstUID = null;
+
+ File indexFile = binaryStore.findIndexFile(aFile, true);
+ if (indexFile != null) {
+ BinaryHeaderAndFooter headAndFoot = binaryStore.readHeaderAndFooter(indexFile);
+ if (headAndFoot != null && headAndFoot.binaryFooter != null) {
+ return headAndFoot.binaryFooter;
+ }
+ }
+ /*
+ * otherwise it would seem that we've a file without a valid end, so unpack it and
+ * get the UID and time of the last item in the file. Can return these in the form of
+ * a BinaryFooter since it's pretty much the same information needed.
+ */
+ BinaryInputStream inputStream = new BinaryInputStream(binaryStore, null);
+ try {
+ // need to work through the file now.
+ if (inputStream.openFile(aFile) == false) {
+ return null;
+ };
+ BinaryObjectData binaryObjectData;
+ BinaryHeader bh = inputStream.readHeader();
+ if (bh==null) {
+ inputStream.closeFile();
+ return null;
+ }
+ int inputFormat = bh.getHeaderFormat();
+ while ((binaryObjectData = inputStream.readNextObject(inputFormat)) != null) {
+ if (binaryObjectData.getTimeMilliseconds() != 0) {
+ lastTime = binaryObjectData.getTimeMilliseconds();
+ }
+ BinaryFooter bf;
+ switch (binaryObjectData.getObjectType()) {
+ case BinaryTypes.FILE_FOOTER:
+ // this is unlikely to happen, since we'll probably already have found an index file.
+ bf = new BinaryFooter();
+ if (bf.readFooterData(binaryObjectData.getDataInputStream(), inputFormat)) {
+ if (bf.getDataDate() != 0) {
+ inputStream.closeFile();
+ return bf;
+ }
+ }
+ break;
+ case BinaryTypes.MODULE_HEADER:
+ break;
+ case BinaryTypes.MODULE_FOOTER:
+ break;
+ case BinaryTypes.DATAGRAM:
+ break;
+ default: // should be data.
+ DataUnitBaseData baseData = binaryObjectData.getDataUnitBaseData();
+ if (baseData != null) {
+ if (baseData.getTimeMilliseconds() != 0) {
+ lastTime = baseData.getTimeMilliseconds();
+ }
+ if (baseData.getUID() != 0) {
+ lastUID = baseData.getUID();
+ if (firstUID == null) {
+ firstUID = lastUID;
+ }
+ }
+ }
+ }
+ }
+ }
+ catch (Exception e) {
+ System.out.printf("Corrupt data file %s: %s\n", aFile, e.getMessage());
+// return null;
+ }
+ try {
+ if (inputStream != null) {
+ inputStream.closeFile();
+ }
+ }
+ catch (Exception e) {
+
+ }
+ if (lastTime != null && lastUID != null) {
+ BinaryFooter bf = new BinaryFooter();
+ bf.setHighestUID(lastUID);
+ bf.setLowestUID(firstUID);
+ bf.setDataDate(lastTime);
+ bf.setFileEndReason(BinaryFooter.END_CRASHED);
+ return bf;
+ }
+ else {
+ return null;
+ }
+ }
+
+ /**
+ * Get the first header. This can be read from a data file whether or not there was a
+ * valid index file created.
+ * @param binFiles
+ * @return
+ */
+ private BinaryHeader findFirstHeader(List binFiles) {
+ BinaryHeader binaryHead = new BinaryHeader();
+ DataInputStream dis = null;
+ for (File aFile : binFiles) {
+ try {
+ dis = new DataInputStream(new BufferedInputStream(new FileInputStream(aFile)));
+ }
+ catch (IOException e) {
+ binaryHead = null;
+ continue;
+ }
+ try {
+ dis.close();
+ }
+ catch (IOException e) {
+
+ }
+ }
+ return binaryHead;
+ }
+
+ /**
+ * first simple status check to see if there are any files there at all.
+ */
+ private int checkStoreStatus() {
+ String currDir = binaryStore.binaryStoreSettings.getStoreLocation();
+ if (currDir == null) {
+ return StoreStatus.STATUS_MISSING;
+ }
+ File currfolder = new File(currDir);
+ if (currfolder.exists() == false) {
+ return StoreStatus.STATUS_MISSING;
+ }
+ // look for files in the folder.
+ boolean hasFiles = hasAnyFiles(currfolder);
+ if (hasFiles) {
+ return StoreStatus.STATUS_HASDATA;
+ }
+ else {
+ return StoreStatus.STATUS_EMPTY;
+ }
+
+ }
+
+ private boolean hasAnyFiles(File currFolder) {
+ PamFileFilter filefilter = new PamFileFilter("data files", ".pgdf");
+ File[] list = currFolder.listFiles(filefilter);
+ if (list == null) {
+ return false;
+ }
+ for (int i = 0; i < list.length; i++) {
+ if (list[i].isDirectory()) {
+ if (hasAnyFiles(list[i])) {
+ return true;
+ }
+ }
+ if (list[i].getAbsolutePath().endsWith(".pgdf")) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+}
diff --git a/src/clickDetector/ClickControl.java b/src/clickDetector/ClickControl.java
index e5b6fce9..5bd48dd9 100644
--- a/src/clickDetector/ClickControl.java
+++ b/src/clickDetector/ClickControl.java
@@ -1086,6 +1086,9 @@ public class ClickControl extends PamControlledUnit implements PamSettings {
}
public ClickDisplayManager getDisplayManager(){
+ if (tabPanelControl == null) {
+ return null;
+ }
return tabPanelControl.clickDisplayManager;
}
diff --git a/src/clickDetector/ClickDetector.java b/src/clickDetector/ClickDetector.java
index db602360..d7330047 100644
--- a/src/clickDetector/ClickDetector.java
+++ b/src/clickDetector/ClickDetector.java
@@ -1809,7 +1809,7 @@ public class ClickDetector extends PamProcess {
newClick.setStartSample(newClick.getStartSample() - firstFilterDelay - secondFilterDelay);
waveDataError = false;
} catch (RawDataUnavailableException e) {
- System.out.println("RawDataUnavailableException");
+ System.out.println(" Click Detector RawDataUnavailableException");
System.out.println(e.getMessage());
}
for (int k = 0; k < nChannels; k++) {
diff --git a/src/clipgenerator/ClipProcess.java b/src/clipgenerator/ClipProcess.java
index e3738243..b61c0a43 100644
--- a/src/clipgenerator/ClipProcess.java
+++ b/src/clipgenerator/ClipProcess.java
@@ -291,7 +291,7 @@ public class ClipProcess extends SpectrogramMarkProcess {
try {
rawData = rawDataBlock.getSamples(startSample, numSamples, channelMap);
} catch (RawDataUnavailableException e) {
- System.out.println(e.getMessage());
+ System.out.println("RawDataUnavailableException in ClipProcess.spectrogramNotification :" + e.getMessage());
return false;
}
if (rawData == null) {
diff --git a/src/cpod/CPODControl.java b/src/cpod/CPODControl.java
index 0df7731a..c095a825 100644
--- a/src/cpod/CPODControl.java
+++ b/src/cpod/CPODControl.java
@@ -19,6 +19,7 @@ import PamController.PamControlledUnitSettings;
import PamController.PamController;
import PamController.PamSettingManager;
import PamController.PamSettings;
+import PamController.fileprocessing.StoreStatus;
import PamView.PamDetectionOverlayGraphics;
import PamView.PamSymbol;
import PamguardMVC.PamDataBlock;
@@ -219,7 +220,7 @@ public class CPODControl extends OfflineFileControl implements PamSettings {
int n = cpodLoader.loadData(dataBlock, usedMapPoints, offlineDataLoadInfo, loadObserver);
return n >= 0;
}
-
+
}
diff --git a/src/d3/D3Control.java b/src/d3/D3Control.java
index 5a462f2c..61c3eff9 100644
--- a/src/d3/D3Control.java
+++ b/src/d3/D3Control.java
@@ -10,6 +10,7 @@ import java.util.List;
import com.opencsv.CSVReader;
import com.opencsv.exceptions.CsvException;
+import PamController.fileprocessing.StoreStatus;
import d3.calibration.CalFileReader;
import d3.calibration.CalibrationInfo;
import d3.calibration.CalibrationSet;
@@ -487,4 +488,5 @@ public class D3Control extends OfflineFileControl {
public D3DataPlotProvider getD3DataPlotProvider() {
return d3DataPlotProvider;
}
+
}
diff --git a/src/dbht/DbHtDialog.java b/src/dbht/DbHtDialog.java
index ce861737..78fbe704 100644
--- a/src/dbht/DbHtDialog.java
+++ b/src/dbht/DbHtDialog.java
@@ -199,7 +199,7 @@ public class DbHtDialog extends PamDialog {
params.calculateFilterThings(currentSampleRate);
}
catch (DbHtException e) {
- System.out.println(e.getMessage());
+ System.out.println("Error in DbHtDialog.updateEverything: " + e.getMessage());
return;
}
double[] f = params.getFilterFrequencies(currentSampleRate);
diff --git a/src/decimator/DecimatorControl.java b/src/decimator/DecimatorControl.java
index 2e4abf4f..7bf9f8ac 100644
--- a/src/decimator/DecimatorControl.java
+++ b/src/decimator/DecimatorControl.java
@@ -45,6 +45,7 @@ import PamController.PamController;
import PamController.PamControllerInterface;
import PamController.PamSettingManager;
import PamController.PamSettings;
+import PamController.fileprocessing.StoreStatus;
import PamguardMVC.PamDataBlock;
import PamguardMVC.PamDataUnit;
import PamguardMVC.PamProcess;
@@ -261,6 +262,6 @@ public class DecimatorControl extends PamControlledUnit implements PamSettings,
double m = fbig % fsmall;
return m == 0;
}
-
+
}
diff --git a/src/difar/DifarProcess.java b/src/difar/DifarProcess.java
index fe1a2886..694c4529 100644
--- a/src/difar/DifarProcess.java
+++ b/src/difar/DifarProcess.java
@@ -1110,7 +1110,7 @@ public class DifarProcess extends PamProcess {
rawData[0] = rawDataAll[0]; // is fine since getSamples was fed a channel map.
}
} catch (RawDataUnavailableException e) {
- System.out.println(e.getMessage());
+ System.out.println("Error in DifarProcess.difarTrigger" + e.getMessage());
return;
}
if (rawData[0] == null) {
diff --git a/src/difar/beamforming/BeamformControl.java b/src/difar/beamforming/BeamformControl.java
index 3ce79544..edf3507b 100644
--- a/src/difar/beamforming/BeamformControl.java
+++ b/src/difar/beamforming/BeamformControl.java
@@ -17,6 +17,7 @@ import PamController.PamControlledUnitSettings;
import PamController.PamController;
import PamController.PamSettingManager;
import PamController.PamSettings;
+import PamController.fileprocessing.StoreStatus;
import PamUtils.PamUtils;
import PamguardMVC.PamDataBlock;
import PamguardMVC.PamProcess;
@@ -197,5 +198,6 @@ public class BeamformControl extends PamControlledUnit implements PamSettings, O
// TODO Auto-generated method stub
return beamformProcess;
}
+
}
\ No newline at end of file
diff --git a/src/difar/demux/NativeDemux.java b/src/difar/demux/NativeDemux.java
index 73a8b9d3..d9edd318 100644
--- a/src/difar/demux/NativeDemux.java
+++ b/src/difar/demux/NativeDemux.java
@@ -98,7 +98,7 @@ public class NativeDemux extends GreenridgeDemux {
return true;
}
catch (UnsatisfiedLinkError e) {
- System.out.println(e.getMessage());
+ System.out.println("Error in NativeDemux.loadLibrary" + e.getMessage());
return false;
}
}
diff --git a/src/generalDatabase/DBControl.java b/src/generalDatabase/DBControl.java
index eb2697d5..525020e7 100644
--- a/src/generalDatabase/DBControl.java
+++ b/src/generalDatabase/DBControl.java
@@ -412,8 +412,9 @@ PamSettingsSource {
}
- if (dbParameters.getUseAutoCommit() == false) {
+// if (dbParameters.getUseAutoCommit() == false) {
JMenuItem commitItem = new JMenuItem("Commit Changes");
+ commitItem.setEnabled(dbParameters.getUseAutoCommit() == false);
commitItem.setToolTipText("Immediately commit recent changes to the database");
commitItem.addActionListener(new ActionListener() {
@Override
@@ -422,7 +423,7 @@ PamSettingsSource {
}
});
menu.add(commitItem);
- }
+// }
if (SMRUEnable.isEnable()) {
JMenuItem speedMenu = new JMenuItem("Test database speed");
diff --git a/src/generalDatabase/DBControlUnit.java b/src/generalDatabase/DBControlUnit.java
index 6baf2658..ff310652 100644
--- a/src/generalDatabase/DBControlUnit.java
+++ b/src/generalDatabase/DBControlUnit.java
@@ -18,12 +18,14 @@ import generalDatabase.backup.DatabaseBackupStream;
import pamScrollSystem.ViewLoadObserver;
import pamViewFX.pamTask.PamTaskUpdate;
import PamController.AWTScheduler;
+import PamController.DataOutputStore;
import PamController.OfflineDataStore;
import PamController.PamControlledUnit;
import PamController.PamController;
import PamController.PamControllerInterface;
import PamController.PamGUIManager;
import PamController.PamSettingManager;
+import PamController.fileprocessing.StoreStatus;
import PamController.status.ModuleStatus;
import PamController.status.QuickRemedialAction;
import PamguardMVC.PamDataBlock;
@@ -39,7 +41,7 @@ import backupmanager.BackupInformation;
* @see DBControl
*
*/
-public class DBControlUnit extends DBControl implements OfflineDataStore {
+public class DBControlUnit extends DBControl implements DataOutputStore {
@@ -492,5 +494,15 @@ public class DBControlUnit extends DBControl implements OfflineDataStore {
return backupInformation;
}
+ @Override
+ public StoreStatus getStoreStatus(boolean getDetail) {
+ return getDbProcess().getStoreStatus(this, getDetail);
+ }
+
+ @Override
+ public boolean deleteDataFrom(long timeMillis) {
+ return getDbProcess().deleteDataFrom(timeMillis);
+ }
+
}
diff --git a/src/generalDatabase/DBProcess.java b/src/generalDatabase/DBProcess.java
index 9e3bf699..167d49f5 100644
--- a/src/generalDatabase/DBProcess.java
+++ b/src/generalDatabase/DBProcess.java
@@ -1,6 +1,9 @@
package generalDatabase;
import generalDatabase.ColumnMetaData.METACOLNAMES;
+import generalDatabase.clauses.FixedClause;
+import generalDatabase.clauses.FromClause;
+import generalDatabase.clauses.PAMSelectClause;
import generalDatabase.pamCursor.PamCursor;
import generalDatabase.ucanAccess.UCanAccessSystem;
@@ -14,6 +17,7 @@ import java.io.InputStreamReader;
import java.io.PrintWriter;
import java.sql.Connection;
import java.sql.DatabaseMetaData;
+import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.ResultSetMetaData;
import java.sql.SQLException;
@@ -21,6 +25,7 @@ import java.sql.SQLWarning;
import java.sql.Statement;
import java.sql.Types;
import java.util.ArrayList;
+import java.util.List;
import javax.swing.JFileChooser;
import javax.swing.JFrame;
@@ -39,6 +44,7 @@ import loggerForms.formdesign.FormEditor;
import PamController.PamControlledUnit;
import PamController.PamController;
import PamController.PamFolders;
+import PamController.fileprocessing.StoreStatus;
import PamUtils.PamCalendar;
import PamUtils.PamFileChooser;
import PamView.dialog.warn.WarnOnce;
@@ -1435,4 +1441,141 @@ public class DBProcess extends PamProcess {
}
}
+ /**
+ * Get the store status for use with pre-process checks.
+ * @param getDetail get full details of start and end times.
+ * @return database store status.
+ */
+ public StoreStatus getStoreStatus(DBControlUnit dbControlUnit, boolean getDetail) {
+ DatabaseStoreStatus dbStoreStatus = new DatabaseStoreStatus(dbControlUnit);
+ // and work out if any tables have anything in them already ...
+ int status = 0;
+ if (dbControlUnit.getConnection() == null) {
+ status = StoreStatus.STATUS_MISSING;
+ }
+ else {
+ boolean anyData = hasAnyOutputData();
+ if (anyData) {
+ status = StoreStatus.STATUS_HASDATA;
+ }
+ else {
+ status = StoreStatus.STATUS_EMPTY;
+ }
+ }
+ if (status == StoreStatus.STATUS_HASDATA && getDetail) {
+ getStoreLimits(dbStoreStatus);
+ }
+ dbStoreStatus.setStoreStatus(status);
+ return dbStoreStatus;
+ }
+
+ private void getStoreLimits(DatabaseStoreStatus dbStoreStatus) {
+ ArrayList allDataBlocks = PamController.getInstance().getDataBlocks();
+ PamTableDefinition tableDefinition;
+ SQLLogging logging;
+
+ // for each datablock, check that the process can log (ignoring GPS process)
+ for (int i = 0; i < allDataBlocks.size(); i++) {
+ PamDataBlock aBlock = allDataBlocks.get(i);
+ logging = aBlock.getLogging();
+ if (logging == null) {
+ continue;
+ }
+ if (aBlock.getMixedDirection() != PamDataBlock.MIX_INTODATABASE) {
+ continue; // don't want things like GPS data.
+ }
+ getStoreLimits(aBlock, dbStoreStatus);
+ }
+
+ }
+
+ /**
+ * Get first and last records for a table.
+ * @param aBlock
+ * @param dbStoreStatus
+ */
+ private void getStoreLimits(PamDataBlock aBlock, DatabaseStoreStatus dbStoreStatus) {
+ // TODO Auto-generated method stub
+ SQLLogging logging = aBlock.getLogging();
+ PamConnection con = databaseControll.getConnection();
+ SQLTypes sqlTypes = con.getSqlTypes();
+ String q1 = String.format("SELECT MIN(UTC) FROM %s",sqlTypes.formatTableName(logging.getTableDefinition().getTableName()));
+ Long t = getUTC(con, q1);
+ dbStoreStatus.testFirstDataTime(t);
+ String q2 = String.format("SELECT MAX(UTC) FROM %s",sqlTypes.formatTableName(logging.getTableDefinition().getTableName()));
+ Long t2 = getUTC(con, q2);
+ dbStoreStatus.testLastDataTime(t2);
+ }
+
+ private Long getUTC(PamConnection con, String qStr) {
+
+ Object utcObject = null;
+ try {
+ PreparedStatement stmt = con.getConnection().prepareStatement(qStr);
+ ResultSet result = stmt.executeQuery();
+ if (result.next()) {
+ utcObject = result.getObject(1);
+
+ }
+ result.close();
+ stmt.close();
+ } catch (SQLException e) {
+ e.printStackTrace();
+ return null;
+ }
+ if (utcObject == null) {
+ return null;
+ }
+ Long millis = SQLTypes.millisFromTimeStamp(utcObject);
+ return millis;
+ }
+
+ /**
+ * Is there any data in any output tables ?
+ * @return
+ */
+ private boolean hasAnyOutputData() {
+ ArrayList allDataBlocks = PamController.getInstance().getDataBlocks();
+ PamTableDefinition tableDefinition;
+ SQLLogging logging;
+
+ // for each datablock, check that the process can log (ignoring GPS process)
+ for (int i = 0; i < allDataBlocks.size(); i++) {
+ PamDataBlock aBlock = allDataBlocks.get(i);
+ logging = aBlock.getLogging();
+ if (logging == null) {
+ continue;
+ }
+ if (aBlock.getMixedDirection() != PamDataBlock.MIX_INTODATABASE) {
+ continue; // don't want things like GPS data.
+ }
+ // get a record count.
+ Integer count = logging.countTableItems(null);
+ if (count != null && count > 0) {
+ return true;
+ }
+ }
+
+ return false;
+ }
+
+ public boolean deleteDataFrom(long timeMillis) {
+ ArrayList allDataBlocks = PamController.getInstance().getDataBlocks();
+ PamTableDefinition tableDefinition;
+ SQLLogging logging;
+
+ // for each datablock, check that the process can log (ignoring GPS process)
+ boolean ok = true;
+ for (int i = 0; i < allDataBlocks.size(); i++) {
+ PamDataBlock aBlock = allDataBlocks.get(i);
+ logging = aBlock.getLogging();
+ if (logging == null) {
+ continue;
+ }
+ PAMSelectClause clause = new FromClause(timeMillis);
+ ok &= logging.deleteData(clause);
+ }
+ return ok;
+ }
+
}
diff --git a/src/generalDatabase/DBSpeedTest.java b/src/generalDatabase/DBSpeedTest.java
index 1f0cecad..d39367c2 100644
--- a/src/generalDatabase/DBSpeedTest.java
+++ b/src/generalDatabase/DBSpeedTest.java
@@ -190,7 +190,7 @@ public class DBSpeedTest {
dropStmt.execute();
} catch (SQLException e) {
// e.printStackTrace();
- System.out.println(e.getMessage());
+ System.out.println("Error in DBSpeedTest testSpeed2:" + e.getMessage());
}
}
try {
diff --git a/src/generalDatabase/DatabaseStoreStatus.java b/src/generalDatabase/DatabaseStoreStatus.java
new file mode 100644
index 00000000..48243930
--- /dev/null
+++ b/src/generalDatabase/DatabaseStoreStatus.java
@@ -0,0 +1,43 @@
+package generalDatabase;
+
+import PamController.fileprocessing.StoreStatus;
+
+public class DatabaseStoreStatus extends StoreStatus {
+
+ private DBControlUnit dbControl;
+
+ public DatabaseStoreStatus(DBControlUnit dbControl) {
+ super(dbControl);
+ this.dbControl = dbControl;
+ }
+
+ @Override
+ public long getFreeSpace() {
+ String name = dbControl.getDatabaseName(); // may not have the path, which is what we need.
+ return getFreeSpace(name); // this may not work, particularly for server based systems.
+ }
+
+ public void testFirstDataTime(Long t) {
+ if (t == null) {
+ return;
+ }
+ if (getFirstDataTime() == null) {
+ setFirstDataTime(t);
+ }
+ if (t < getFirstDataTime()) {
+ setFirstDataTime(t);
+ }
+ }
+ public void testLastDataTime(Long t) {
+ if (t == null) {
+ return;
+ }
+ if (getLastDataTime() == null) {
+ setLastDataTime(t);
+ }
+ if (t > getLastDataTime()) {
+ setLastDataTime(t);
+ }
+ }
+
+}
diff --git a/src/generalDatabase/PamTableDefinition.java b/src/generalDatabase/PamTableDefinition.java
index 6456671d..e8a2b5dc 100644
--- a/src/generalDatabase/PamTableDefinition.java
+++ b/src/generalDatabase/PamTableDefinition.java
@@ -13,7 +13,7 @@ import PamUtils.PamCalendar;
* tables. Also used to prepare Sql statements for writing and
* reading back data.
*
- * I did a bit of redifining what columns are used for on 4 Oct, 2012.
+ * I did a bit of redefining what columns are used for on 4 Oct, 2012.
* PCLocalTime was a UTC time from the PC of the time analysis took place.
* When running in real time, this would be the same as the data in the UTC column
* (give or take the odd second for data to get through the system). I've now defined
diff --git a/src/generalDatabase/PamTableItem.java b/src/generalDatabase/PamTableItem.java
index dc523083..2a046623 100644
--- a/src/generalDatabase/PamTableItem.java
+++ b/src/generalDatabase/PamTableItem.java
@@ -225,7 +225,10 @@ public class PamTableItem implements Cloneable {
// }
public String getDeblankedStringValue() {
- if (sqlType != Types.CHAR || value == null) {
+// if (sqlType != Types.CHAR || value == null) {
+// return null;
+// }
+ if (value instanceof String == false) {
return null;
}
return ((String) value).trim();
@@ -485,6 +488,15 @@ public class PamTableItem implements Cloneable {
if (value == null) {
return Float.NaN;
}
+ if (value instanceof String) {
+ try {
+ double val = Double.valueOf((String) value);
+ return (float) val;
+ }
+ catch (NumberFormatException ex) {
+ return Float.NaN;
+ }
+ }
if (value.getClass() == Double.class) {
double dVal = (Double) value;
return (float) dVal;
diff --git a/src/generalDatabase/SQLLogging.java b/src/generalDatabase/SQLLogging.java
index 7cbd086d..8e4cc4e5 100644
--- a/src/generalDatabase/SQLLogging.java
+++ b/src/generalDatabase/SQLLogging.java
@@ -635,11 +635,20 @@ public abstract class SQLLogging {
if (pamConn == null) {
return null;
}
+
SQLTypes sqlTypes = pamConn.getSqlTypes();
//the clause contains 'WHERE' so it's possible to make a null one.
- String qStr = String.format("SELECT COUNT(%s.Id) FROM %s %s",
- pamTableDefinition.getTableName(),
- pamViewParameters.getSelectClause(sqlTypes));
+ String qStr;
+ if (pamViewParameters == null) {
+ qStr = String.format("SELECT COUNT(Id) FROM %s",
+ pamTableDefinition.getTableName());
+ }
+ else {
+ qStr = String.format("SELECT COUNT(%s.Id) FROM %s %s",
+ pamTableDefinition.getTableName(),
+ pamTableDefinition.getTableName(),
+ pamViewParameters.getSelectClause(sqlTypes));
+ }
int count = 0;
try {
PreparedStatement stmt = pamConn.getConnection().prepareStatement(qStr);
diff --git a/src/generalDatabase/sqlite/SqliteSystem.java b/src/generalDatabase/sqlite/SqliteSystem.java
index b41d71b8..22a1bc61 100644
--- a/src/generalDatabase/sqlite/SqliteSystem.java
+++ b/src/generalDatabase/sqlite/SqliteSystem.java
@@ -103,7 +103,10 @@ public class SqliteSystem extends DBSystem implements PamSettings {
if (commandName == null) {
return;
}
- setDatabaseName(commandName);
+ File commandFile = new File(commandName);
+ // check the file end is of the right type. Some batch systems may not get this right.
+ commandFile = PamFileFilter.checkFileEnd(commandFile, "sqlite3", true);
+ setDatabaseName(commandFile.getAbsolutePath());
}
/**
@@ -276,7 +279,9 @@ public class SqliteSystem extends DBSystem implements PamSettings {
{
if(connection != null){
// if (USEAUTOCOMMIT == false) {
- connection.getConnection().commit();
+ if (connection.getConnection().getAutoCommit()) {
+ connection.getConnection().commit();
+ }
// }
connection.getConnection().close();
}
@@ -303,6 +308,30 @@ public class SqliteSystem extends DBSystem implements PamSettings {
return true;
}
+ @Override
+ public boolean checkDatabaseExists(String dbName) {
+ String commandName = GlobalArguments.getParam(DBControl.GlobalDatabaseNameArg);
+ if (commandName != null) {
+ return checkCommandLineDatabase();
+ }
+ return super.checkDatabaseExists(dbName);
+ }
+
+ private boolean checkCommandLineDatabase() {
+ String commandName = GlobalArguments.getParam(DBControl.GlobalDatabaseNameArg);
+ if (commandName == null) {
+ return false;
+ }
+ File dbFile = new File(commandName);
+ dbFile = PamFileFilter.checkFileEnd(dbFile, ".sqlite3", true);
+ commandName = dbFile.getAbsolutePath();
+ if (dbFile.exists() == false) {
+ // create a new database without asking.
+ createNewDatabase(commandName);
+ }
+ return dbFile.exists();
+ }
+
@Override
public String getDatabaseName() {
/*
@@ -311,6 +340,9 @@ public class SqliteSystem extends DBSystem implements PamSettings {
*/
String commandName = GlobalArguments.getParam(DBControl.GlobalDatabaseNameArg);
if (commandName != null) {
+ File dbFile = new File(commandName);
+ dbFile = PamFileFilter.checkFileEnd(dbFile, ".sqlite3", true);
+ commandName = dbFile.getAbsolutePath();
return commandName;
}
diff --git a/src/gpl/contour/ContourFinder.java b/src/gpl/contour/ContourFinder.java
index 7acc7476..ea89b9a4 100644
--- a/src/gpl/contour/ContourFinder.java
+++ b/src/gpl/contour/ContourFinder.java
@@ -261,7 +261,13 @@ public class ContourFinder {
* Note that binData is padded, whiteData isn't.
*/
GPLContour newContour = new GPLContour(iT-1, iF-1+gplProcess.binLo, whiteData[iT-1][iF-1], energyData[iT-1][iF-1]);
- findAdjacentPoints(newContour, iT, iF, binData, whiteData, energyData, con);
+ try {
+ findAdjacentPoints(newContour, iT, iF, binData, whiteData, energyData, con);
+ }
+ catch (StackOverflowError e) {
+ System.out.println("Stack overflow in GPLContour.findRegion for bloated contour size " + newContour.getArea());
+ System.out.println("The contour is incomplete, but has not been discarded");
+ }
return newContour;
}
@@ -282,6 +288,10 @@ public class ContourFinder {
* Separate loops to get above and to sides, NOT diagonal matches.
* could also easily change to connect 8 instead of connect 4.
*/
+ if (newContour.getArea() >= 500) {
+ System.out.println("GPL Contour size exceeds maximum value of 500 points so ceasing to grow it");
+ return;
+ }
binData[currT][currF] = 0; // set current point to 0 so it doesn't get found again
int nX = binData.length-1;
int nY = binData[0].length-1;
diff --git a/src/group3dlocaliser/algorithm/hyperbolic/HyperbolicLocaliser.java b/src/group3dlocaliser/algorithm/hyperbolic/HyperbolicLocaliser.java
index 5a4884cf..dc6e5eb4 100644
--- a/src/group3dlocaliser/algorithm/hyperbolic/HyperbolicLocaliser.java
+++ b/src/group3dlocaliser/algorithm/hyperbolic/HyperbolicLocaliser.java
@@ -217,7 +217,7 @@ public class HyperbolicLocaliser extends TOADBaseAlgorithm {
// answer2 = rightMatrix.times(leftInverse.transpose());
}
catch (Exception e) {
- System.out.println(e.getMessage());
+ System.out.println("Error in HyperbolicLocaliser.processTOADsPlane: " + e.getMessage());
// Matrix m = leftMatrix.transpose();
// m = m.inverse();
return null;
@@ -254,7 +254,7 @@ public class HyperbolicLocaliser extends TOADBaseAlgorithm {
}
catch (Exception e) {
- System.out.println(e.getMessage());
+ System.out.println("Error2 in HyperbolicLocaliser.processTOADsPlane: " + e.getMessage());
}
// now need to rotate back from the plane that we're in to get back to the original position.
@@ -736,7 +736,7 @@ public class HyperbolicLocaliser extends TOADBaseAlgorithm {
}
catch (Exception e) {
- System.out.println(e.getMessage());
+ System.out.println("Error in HyperbolicLocaliser.processTOADs3D: " + e.getMessage());
}
LatLong pos = geometry.getReferenceGPS().addDistanceMeters(centre.getCoordinate(0)+answer.get(0,0),
diff --git a/src/group3dlocaliser/algorithm/toadsimplex/ToadSimplexLocaliser.java b/src/group3dlocaliser/algorithm/toadsimplex/ToadSimplexLocaliser.java
index e16db196..d362a8ef 100644
--- a/src/group3dlocaliser/algorithm/toadsimplex/ToadSimplexLocaliser.java
+++ b/src/group3dlocaliser/algorithm/toadsimplex/ToadSimplexLocaliser.java
@@ -146,7 +146,7 @@ public class ToadSimplexLocaliser extends TOADBaseAlgorithm {
try {
result = optimiser.optimize(chiFunc, goal, start);
} catch (OptimizationException | FunctionEvaluationException | IllegalArgumentException e) {
- System.out.println(e.getMessage());
+ System.out.println("Error in TOADSimplexLocaliser.processTOADs: " + e.getMessage());
nFails++;
System.out.printf("%d fails out of %d = %3.2f%%\n", nFails, nCalls, (double) nFails * 100. / nCalls);
resultChiData[iStart] = null;
diff --git a/src/networkTransfer/emulator/EmulatorStream.java b/src/networkTransfer/emulator/EmulatorStream.java
index 088e2d65..a62d19a3 100644
--- a/src/networkTransfer/emulator/EmulatorStream.java
+++ b/src/networkTransfer/emulator/EmulatorStream.java
@@ -129,7 +129,7 @@ public class EmulatorStream {
try {
emSocket.getOutputStream().write(data);
} catch (IOException e) {
- System.out.println(e.getMessage());
+ System.out.println("Error in EmulatorStream.sendPamCommand: " + e.getMessage());
}
}
private boolean openSocket() {
@@ -137,9 +137,9 @@ public class EmulatorStream {
try {
emSocket = new Socket(sktParams.ipAddress, sktParams.portNumber);
} catch (UnknownHostException e) {
- System.out.println(e.getMessage());
+ System.out.println("UnknownHostException in EmulatorStream.sendPamCommand: " + e.getMessage());
} catch (IOException e) {
- System.out.println(e.getMessage());
+ System.out.println("IOException in EmulatorStream.sendPamCommand: " + e.getMessage());
}
if (emSocket != null) {
status.socketStatus = emSocket.isConnected();
diff --git a/src/networkTransfer/receive/NetworkReceiver.java b/src/networkTransfer/receive/NetworkReceiver.java
index 67d5a5cb..3bb9fc02 100644
--- a/src/networkTransfer/receive/NetworkReceiver.java
+++ b/src/networkTransfer/receive/NetworkReceiver.java
@@ -458,7 +458,7 @@ public class NetworkReceiver extends PamControlledUnit implements PamSettings, N
// checkExistingThreads(clientSocket, rxThread);
}
} catch (IOException e) {
- System.out.println(e.getMessage());
+ System.out.println("IOException in NeetworkReceiver" + e.getMessage());
}
}
diff --git a/src/networkTransfer/send/NetworkObjectPacker.java b/src/networkTransfer/send/NetworkObjectPacker.java
index 08b9aa0d..ee3aac88 100644
--- a/src/networkTransfer/send/NetworkObjectPacker.java
+++ b/src/networkTransfer/send/NetworkObjectPacker.java
@@ -75,7 +75,10 @@ public class NetworkObjectPacker {
BinaryDataSource binarySource = dataBlock.getBinaryDataSource();
int dataType2 = dataBlock.getQuickId();
- BinaryObjectData packedObject = binarySource.getPackedData(dataUnit);
+ BinaryObjectData packedObject = null;
+ synchronized (BinaryDataSource.packSynchObject) {
+ packedObject = binarySource.getPackedData(dataUnit);
+ }
byte[] data = packedObject.getData();
int duDataLength = data.length + 12;
DataUnitBaseData baseData = dataUnit.getBasicData();
diff --git a/src/networkTransfer/send/NetworkSender.java b/src/networkTransfer/send/NetworkSender.java
index a5a3b011..65f2774b 100644
--- a/src/networkTransfer/send/NetworkSender.java
+++ b/src/networkTransfer/send/NetworkSender.java
@@ -23,10 +23,12 @@ import javax.swing.SwingWorker;
import networkTransfer.emulator.NetworkEmulator;
import networkTransfer.receive.NetworkReceiver;
+import pamguard.GlobalArguments;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
+import Acquisition.FolderInputSystem;
import PamController.PamControlledUnit;
import PamController.PamControlledUnitSettings;
import PamController.PamController;
@@ -44,6 +46,11 @@ import PamguardMVC.PamDataBlock;
*
*/
public class NetworkSender extends PamControlledUnit implements PamSettings {
+
+ public static final String ADDRESS = "-netSend.address";
+ public static final String PORT = "-netSend.port";
+ public static final String ID1 = "-netSend.id1";
+ public static final String ID2 = "-netSend.id2";
protected NetworkSendParams networkSendParams = new NetworkSendParams();
private NetworkEmulator networkEmulator;
@@ -169,6 +176,28 @@ public class NetworkSender extends PamControlledUnit implements PamSettings {
public boolean restoreSettings(
PamControlledUnitSettings pamControlledUnitSettings) {
networkSendParams = ((NetworkSendParams) pamControlledUnitSettings.getSettings()).clone();
+
+ String address = GlobalArguments.getParam(ADDRESS);
+ String portString = GlobalArguments.getParam(PORT);
+ String id1String = GlobalArguments.getParam(ID1);
+ String id2String = GlobalArguments.getParam(ID2);
+
+ if (address != null) {
+ networkSendParams.ipAddress = address; // remember it.
+ }
+
+ if(portString != null) {
+ networkSendParams.portNumber = Integer.valueOf(portString);
+ }
+
+ if(id1String!=null) {
+ networkSendParams.stationId1 = Integer.valueOf(id1String);
+ }
+
+ if(id2String!=null) {
+ networkSendParams.stationId1 = Integer.valueOf(id2String);
+ }
+
return (networkSendParams != null);
}
@@ -452,7 +481,7 @@ public class NetworkSender extends PamControlledUnit implements PamSettings {
return true;
} catch (IOException e) {
// e.printStackTrace();
- System.out.println(e.getMessage());
+ System.out.println("IOException in NeetworkSender.writeByteData: " + e.getMessage());
currStatus = "Socket Closed";
return false;
}
@@ -465,7 +494,7 @@ public class NetworkSender extends PamControlledUnit implements PamSettings {
return true;
} catch (IOException e) {
// e.printStackTrace();
- System.out.println(e.getMessage());
+ System.out.println("IOException in NeetworkSender.writeStringData" + e.getMessage());
currStatus = "Socket Closed";
return false;
}
@@ -513,7 +542,7 @@ public class NetworkSender extends PamControlledUnit implements PamSettings {
tcpWriter.close();
} catch (IOException e) {
// e.printStackTrace();
- System.out.println(e.getMessage());
+ System.out.println("IOException in NeetworkSender.closeconnection" + e.getMessage());
}
tcpSocket = null;
tcpWriter = null;
diff --git a/src/nidaqdev/NIFilePlayback.java b/src/nidaqdev/NIFilePlayback.java
index f87a35ef..3b8b42ed 100644
--- a/src/nidaqdev/NIFilePlayback.java
+++ b/src/nidaqdev/NIFilePlayback.java
@@ -127,7 +127,8 @@ public class NIFilePlayback implements FilePlaybackDevice, PamSettings {
@Override
public boolean preparePlayback(PlaybackParameters playbackParameters) {
- if (niDeviceLUT == null || niDeviceLUT.length <= playbackParameters.deviceNumber) {
+ if (niDeviceLUT == null || niDeviceLUT.length <= playbackParameters.deviceNumber
+ || playbackParameters.deviceNumber < 0) {
return false;
}
int bn = niDeviceLUT[playbackParameters.deviceNumber];
diff --git a/src/nmeaEmulator/SerialOutput.java b/src/nmeaEmulator/SerialOutput.java
index bff19830..4ec93884 100644
--- a/src/nmeaEmulator/SerialOutput.java
+++ b/src/nmeaEmulator/SerialOutput.java
@@ -68,7 +68,7 @@ public class SerialOutput {
SerialPort.NO_PARITY,
SerialPort.FLOW_CONTROL_DISABLED); // disable flow control, since this is the RXTX default and it's not specified above
} catch (PJSerialException e) {
- System.out.println(e.getMessage());
+ System.out.println("PJSerialException in SerialOutput: " + e.getMessage());
e.printStackTrace();
return false;
}
diff --git a/src/offlineProcessing/DataCopyTask.java b/src/offlineProcessing/DataCopyTask.java
index 40071cdf..c67a00c9 100644
--- a/src/offlineProcessing/DataCopyTask.java
+++ b/src/offlineProcessing/DataCopyTask.java
@@ -21,8 +21,6 @@ import dataMap.OfflineDataMapPoint;
* @param
*/
public class DataCopyTask extends OfflineTask {
-
- private PamDataBlock pamDataBlock;
private SQLLogging sqlLogging;
@@ -40,7 +38,6 @@ public class DataCopyTask extends OfflineTask {
*/
public DataCopyTask(PamDataBlock pamDataBlock) {
super(pamDataBlock);
- this.pamDataBlock = pamDataBlock;
this.sqlLogging = pamDataBlock.getLogging();
this.binaryDataSource = pamDataBlock.getBinaryDataSource();
setParentDataBlock(pamDataBlock);
@@ -77,7 +74,7 @@ public class DataCopyTask extends OfflineTask {
@Override
public String getName() {
- return "Copy " + pamDataBlock.getDataName() + " To database";
+ return "Copy " + getDataBlock().getDataName() + " To database";
}
@Override
diff --git a/src/offlineProcessing/OfflineTask.java b/src/offlineProcessing/OfflineTask.java
index a1dc9e15..7f3e0ec1 100644
--- a/src/offlineProcessing/OfflineTask.java
+++ b/src/offlineProcessing/OfflineTask.java
@@ -458,5 +458,4 @@ public abstract class OfflineTask {
}
-
}
diff --git a/src/pamViewFX/fxNodes/pamAxis/PamAxisPane2.java b/src/pamViewFX/fxNodes/pamAxis/PamAxisPane2.java
index 905109bc..1a84a220 100644
--- a/src/pamViewFX/fxNodes/pamAxis/PamAxisPane2.java
+++ b/src/pamViewFX/fxNodes/pamAxis/PamAxisPane2.java
@@ -250,7 +250,7 @@ public class PamAxisPane2 extends StackPane {
// }
}
catch (Exception e) {
- System.out.println(e.getMessage());
+ System.out.println("Exception in PamAxisPanel2.ScaledNumberFormatter: " + e.getMessage());
e.printStackTrace();
return "Err";
}
diff --git a/src/pamguard/Pamguard.java b/src/pamguard/Pamguard.java
index 4a142c0b..a80c16cc 100644
--- a/src/pamguard/Pamguard.java
+++ b/src/pamguard/Pamguard.java
@@ -29,6 +29,7 @@ import PamController.PamGUIManager;
import PamController.PamSettingManager;
import PamController.PamguardVersionInfo;
import PamController.pamBuoyGlobals;
+import PamController.fileprocessing.ReprocessStoreChoice;
import PamModel.SMRUEnable;
import PamUtils.FileFunctions;
import PamUtils.PamExceptionHandler;
@@ -42,6 +43,7 @@ import PamguardMVC.debug.Debug;
import binaryFileStorage.BinaryStore;
import dataPlotsFX.JamieDev;
import generalDatabase.DBControl;
+import networkTransfer.send.NetworkSender;
import rocca.RoccaDev;
import java.io.BufferedReader;
@@ -121,6 +123,7 @@ public class Pamguard {
int runMode = PamController.RUN_NORMAL;
String InputPsf = "NULL";
+
// set up the system to output to both a log file and the console window. Also
@@ -136,6 +139,14 @@ public class Pamguard {
// TimeZone.setDefault(PamCalendar.defaultTimeZone);
System.out.println("**********************************************************");
+ // print out the entire command line
+ if (args != null && args.length > 0) {
+ System.out.printf("Command line options: ");
+ for (int i = 0; i < args.length; i++) {
+ System.out.printf("\"%s\" ", args[i]);
+ }
+ System.out.printf("\n");
+ }
try {
// get the java runnable file name.
// http://stackoverflow.com/questions/4294522/jar-file-name-form-java-code
@@ -209,22 +220,25 @@ public class Pamguard {
}
else if (anArg.equalsIgnoreCase("-nogui")) {
PamGUIManager.setType(PamGUIManager.NOGUI);
+ System.out.println("no gui operation.");
}
- ///////////////
- else if (anArg.equalsIgnoreCase("-psf")) {
+ else if (anArg.equalsIgnoreCase("-psf") || anArg.equalsIgnoreCase("-psfx")) {
String autoPsf = args[iArg++];
PamSettingManager.remote_psf = autoPsf;
System.out.println("Running using settings from " + autoPsf);
}
else if (anArg.equalsIgnoreCase("-port")) {
// port id to open a udp port to receive commands
- pamBuoyGlobals.setNetworkControlPort(Integer.parseInt(args[iArg++]));
+ String port = args[iArg++];
+ pamBuoyGlobals.setNetworkControlPort(Integer.parseInt(port));
+ System.out.println("Setting UDP control port " + port);
}
- else if (anArg.equalsIgnoreCase("-mport")) {
+ else if (anArg.equalsIgnoreCase("-multicast") || anArg.equalsIgnoreCase("-mport")) {
// multicast control (for multiple PAMGuards)
String mAddr = args[iArg++];
int mPort = Integer.parseInt(args[iArg++]);
pamBuoyGlobals.setMultiportConfig(mAddr, mPort);
+ System.out.printf("Setting multicast control addr %s port %d\n", mAddr, mPort);
}
else if (anArg.equalsIgnoreCase("-nolog")) {
System.out.println("Disabling log file from command line switch...");
@@ -232,23 +246,56 @@ public class Pamguard {
}
else if (anArg.equalsIgnoreCase(BinaryStore.GlobalFolderArg)) {
// output folder for binary files.
- GlobalArguments.setParam(BinaryStore.GlobalFolderArg, args[iArg++]);
+ String binFolder = args[iArg++];
+ GlobalArguments.setParam(BinaryStore.GlobalFolderArg, binFolder);
+ System.out.println("Setting output folder for binary files to " + binFolder);
}
else if (anArg.equalsIgnoreCase(DBControl.GlobalDatabaseNameArg)) {
// database file name
- GlobalArguments.setParam(DBControl.GlobalDatabaseNameArg, args[iArg++]);
+ String dbName = args[iArg++];
+ GlobalArguments.setParam(DBControl.GlobalDatabaseNameArg, dbName);
+ System.out.println("Setting output database file to " + dbName);
}
else if (anArg.equalsIgnoreCase(FolderInputSystem.GlobalWavFolderArg)) {
// source folder for wav files (or other supported sound files)
- GlobalArguments.setParam(FolderInputSystem.GlobalWavFolderArg, args[iArg++]);
+ String wavFolder = args[iArg++];
+ GlobalArguments.setParam(FolderInputSystem.GlobalWavFolderArg, wavFolder);
+ System.out.println("Setting input wav file folder to " + wavFolder);
}
else if (anArg.equalsIgnoreCase(PamController.AUTOSTART)) {
// auto start processing.
GlobalArguments.setParam(PamController.AUTOSTART, PamController.AUTOSTART);
+ System.out.println("Setting autostart ON");
}
else if (anArg.equalsIgnoreCase(PamController.AUTOEXIT)) {
// auto exit at end of processing.
GlobalArguments.setParam(PamController.AUTOEXIT, PamController.AUTOEXIT);
+ System.out.println("Setting autoexit ON");
+ }
+ else if (anArg.equalsIgnoreCase(NetworkSender.ADDRESS)) {
+ // auto exit at end of processing.
+ GlobalArguments.setParam(NetworkSender.ADDRESS, args[iArg++]);
+ }
+ else if (anArg.equalsIgnoreCase(NetworkSender.ID1)) {
+ // auto exit at end of processing.
+ GlobalArguments.setParam(NetworkSender.ID1, args[iArg++]);
+ }
+ else if (anArg.equalsIgnoreCase(NetworkSender.ID2)) {
+ // auto exit at end of processing.
+ GlobalArguments.setParam(NetworkSender.ID2, args[iArg++]);
+ }
+ else if (anArg.equalsIgnoreCase(NetworkSender.PORT)) {
+ // auto exit at end of processing.
+ GlobalArguments.setParam(NetworkSender.PORT, args[iArg++]);
+ }
+ else if (anArg.equalsIgnoreCase(ReprocessStoreChoice.paramName)) {
+ String arg = args[iArg++];
+ ReprocessStoreChoice choice = ReprocessStoreChoice.valueOf(arg);
+ if (choice == null) {
+ String warn = String.format("Reprocessing storage input parameter %s value \"%s\" is not a recognised value", ReprocessStoreChoice.paramName, arg);
+ WarnOnce.showWarning("Invalid input parameter", warn, WarnOnce.WARNING_MESSAGE);
+ }
+ GlobalArguments.setParam(ReprocessStoreChoice.paramName, arg);
}
else if (anArg.equalsIgnoreCase("-help")) {
System.out.println("--PamGuard Help");
diff --git a/src/rocca/RoccaContour.java b/src/rocca/RoccaContour.java
index cce19f5a..81ba39f7 100644
--- a/src/rocca/RoccaContour.java
+++ b/src/rocca/RoccaContour.java
@@ -192,7 +192,7 @@ public class RoccaContour {
(currentStartSample, fftLength, channelMap );
}
catch (RawDataUnavailableException e) {
- System.out.println(e.getMessage());
+ System.out.println("RawDataUnavailableException in ROCCAContour.generateContour: " + e.getMessage());
currentRawData = null;
}
/* every now and then the FFTDataUnit start sample doesn't match up
@@ -237,7 +237,7 @@ public class RoccaContour {
nextRawData = rawDataBlockIn.getSamples(nextStartSample, fftLength, channelMap );
}
catch (RawDataUnavailableException e) {
- System.out.println(e.getMessage());
+ System.out.println("RawDataUnavailableException (2) in ROCCAContour.generateContour: " + e.getMessage());
nextRawData = null;
}
nextFFT = nextFFTDataUnit.getFftData();
diff --git a/src/rocca/RoccaSpecPopUp.java b/src/rocca/RoccaSpecPopUp.java
index 57dfa487..7dc49431 100644
--- a/src/rocca/RoccaSpecPopUp.java
+++ b/src/rocca/RoccaSpecPopUp.java
@@ -2333,7 +2333,7 @@ public class RoccaSpecPopUp extends javax.swing.JPanel {
}
}
catch (RawDataUnavailableException e) {
- System.out.println(e.getMessage());
+ System.out.println("RawDataUnavailableException in ROCCASpecPopUp: " + e.getMessage());
}
if (rawDataValues==null) {
diff --git a/src/rockBlock/RockBlockProcess2.java b/src/rockBlock/RockBlockProcess2.java
index 94acda03..3cd0d17a 100644
--- a/src/rockBlock/RockBlockProcess2.java
+++ b/src/rockBlock/RockBlockProcess2.java
@@ -280,7 +280,7 @@ public class RockBlockProcess2 extends PamProcess implements ModuleStatusManager
timeDelays[0] = rockBlockControl.getParams().getCommTiming();
initializeRockBlock();
} catch (Exception e) {
- System.out.println(e.getMessage());
+ System.out.println("Exception in rockBlockProcess: " + e.getMessage());
rbStatus.setPortError(e.getMessage());
rbStatus.setCommReady(false);
commReady = false;
diff --git a/src/soundtrap/DWVConverter.java b/src/soundtrap/DWVConverter.java
index ca4ba9d0..a476d26f 100644
--- a/src/soundtrap/DWVConverter.java
+++ b/src/soundtrap/DWVConverter.java
@@ -219,7 +219,7 @@ public class DWVConverter {
}
}
catch (Exception e) {
- System.out.println(e.getMessage());
+ System.out.println("Exception 1 in DWVConverter.effortLine: " + e.getMessage());
}
binaryStream.openOutputFiles(bclLine.getMilliseconds());
binaryStream.writeHeader(bclLine.getMilliseconds(), System.currentTimeMillis());
@@ -234,7 +234,7 @@ public class DWVConverter {
PamCalendar.formatDBDateTime(wavFileStop), PamCalendar.formatDBDateTime(bclLine.getMilliseconds()));
}
catch (Exception e) {
- System.out.println(e.getMessage());
+ System.out.println("Exception 2 in DWVConverter.effortLine: " + e.getMessage());
}
binaryStream.writeModuleFooter();
binaryStream.writeFooter(bclLine.getMilliseconds(), System.currentTimeMillis(), BinaryFooter.END_UNKNOWN);
diff --git a/src/soundtrap/ImportBCLDialog.java b/src/soundtrap/ImportBCLDialog.java
index a0396d8c..3de2e6ee 100644
--- a/src/soundtrap/ImportBCLDialog.java
+++ b/src/soundtrap/ImportBCLDialog.java
@@ -453,12 +453,18 @@ public class ImportBCLDialog extends PamDialog {
int nFiles = 0;
int nCDET = 0;
int nDWV = 0;
+
+ STClickControl clickControl = (STClickControl) PamController.getInstance().findControlledUnit(STClickControl.STUNITTYPE, stToolsParams.clickDetName);
+
for (File xmlFile:xmlFiles) {
Debug.out.println("Opening xml file " + xmlFile.getAbsolutePath());
- if (xmlFile.getAbsolutePath().equals("E:\\STOctober2016\\335839252\\335839252.161031002807.log.xml")) {
- System.out.println("Opening xml file " + xmlFile.getAbsolutePath());
- }
+// if (xmlFile.getAbsolutePath().equals("E:\\STOctober2016\\335839252\\335839252.161031002807.log.xml")) {
+// System.out.println("Opening xml file " + xmlFile.getAbsolutePath());
+// }
STXMLFile xmlFileInfo = STXMLFile.openXMLFile(xmlFile, customDateTimeFormat.getText());
+ if (clickControl != null && xmlFileInfo.getSudDetectorInfo() != null) {
+ clickControl.setSudClickDetectorInfo(xmlFileInfo.getSudDetectorInfo());
+ }
if (xmlFileInfo == null || xmlFileInfo.getDwvInfo() == null) {
String title = "Error with Soundtrap file";
diff --git a/src/soundtrap/STAcquisitionControl.java b/src/soundtrap/STAcquisitionControl.java
index 09eebc77..bfad1da2 100644
--- a/src/soundtrap/STAcquisitionControl.java
+++ b/src/soundtrap/STAcquisitionControl.java
@@ -28,6 +28,7 @@ import java.awt.Component;
import Acquisition.AcquisitionControl;
import Acquisition.DaqSystem;
+import PamController.PamControlledUnitSettings;
/**
* @author SCANS
@@ -102,6 +103,18 @@ public class STAcquisitionControl extends AcquisitionControl {
return stDaqSystem;
}
+ @Override
+ public boolean restoreSettings(PamControlledUnitSettings pamControlledUnitSettings) {
+ // TODO Auto-generated method stub
+ boolean ok = super.restoreSettings(pamControlledUnitSettings);
+ if (Float.isNaN(this.getAcquisitionParameters().sampleRate)) {
+ System.out.println("Nan sample rate read back in sound trap data control. ");
+ this.getAcquisitionParameters().sampleRate = 384000.f;
+ }
+
+ return ok;
+ }
+
}
diff --git a/src/soundtrap/STAcquisitionProcess.java b/src/soundtrap/STAcquisitionProcess.java
index caf2583a..1b99ef31 100644
--- a/src/soundtrap/STAcquisitionProcess.java
+++ b/src/soundtrap/STAcquisitionProcess.java
@@ -87,6 +87,12 @@ public class STAcquisitionProcess extends AcquisitionProcess {
}
}
}
+
+ @Override
+ public float getSampleRate() {
+ float fs = super.getSampleRate();
+ return fs;
+ }
@Override
/**
@@ -150,6 +156,7 @@ public class STAcquisitionProcess extends AcquisitionProcess {
public boolean isStalled() {
return false;
}
+
diff --git a/src/soundtrap/STClickControl.java b/src/soundtrap/STClickControl.java
index 0f251106..203357fe 100644
--- a/src/soundtrap/STClickControl.java
+++ b/src/soundtrap/STClickControl.java
@@ -25,19 +25,29 @@
package soundtrap;
import java.awt.Frame;
+import java.awt.event.ActionEvent;
+import java.awt.event.ActionListener;
+import java.io.Serializable;
import java.util.ArrayList;
import javax.swing.JMenu;
import javax.swing.JMenuItem;
+import javax.swing.JSeparator;
+
+import org.pamguard.x3.sud.SUDClickDetectorInfo;
import Acquisition.AcquisitionControl;
-import Acquisition.sud.SUDNotificationManager;
+import PamController.PamControlledUnitSettings;
import PamController.PamController;
+import PamController.PamSettingManager;
+import PamController.PamSettings;
import PamguardMVC.PamRawDataBlock;
import clickDetector.ClickBTDisplay;
import clickDetector.ClickControl;
import clickDetector.ClickDisplay;
import clickDetector.ClickDisplayManager;
+import clickDetector.ClickParameters;
+import soundtrap.sud.SUDParamsDialog;
import soundtrap.sud.SudFileDWVHandler;
/**
@@ -46,6 +56,8 @@ import soundtrap.sud.SudFileDWVHandler;
*/
public class STClickControl extends ClickControl {
+ private SUDClickDetectorInfo sudClickDetectorInfo;
+
public static final String STUNITTYPE = "SoundTrap Click Detector";
/**
@@ -67,6 +79,8 @@ public class STClickControl extends ClickControl {
sudFileDWVHandler = new SudFileDWVHandler(this);
sudFileDWVHandler.subscribeSUD();
+
+ PamSettingManager.getInstance().registerSettings(new SUDSettings());
}
@Override
@@ -117,9 +131,23 @@ public class STClickControl extends ClickControl {
}
}
+ JMenuItem sudItem = new JMenuItem("Sound Trap settings ...");
+ sudItem.addActionListener(new ActionListener() {
+ @Override
+ public void actionPerformed(ActionEvent e) {
+ showSudParameters(parentFrame);
+ }
+ });
+ newMenu.add(sudItem, 0);
+ newMenu.add(new JSeparator(), 1);
+
return newMenu;
}
+ protected void showSudParameters(Frame parentFrame) {
+ SUDParamsDialog.showDialog(parentFrame, this);
+ }
+
@Override
public void pamStart() {
sudFileDWVHandler.pamStart();
@@ -146,6 +174,10 @@ public class STClickControl extends ClickControl {
*/
public void updateDisplayScrollers(long timeMillis) {
ClickDisplayManager dispManager = getDisplayManager();
+ if (dispManager == null) {
+ // happens in -nogui operation.
+ return;
+ }
ArrayList dispList = dispManager.getWindowList();
for (ClickDisplay display : dispList) {
if (display instanceof ClickBTDisplay) {
@@ -153,6 +185,89 @@ public class STClickControl extends ClickControl {
}
}
}
+
+ // this was a bad idea since we need to keep hold of settings for the
+ // classifier, which are in with the main set ...
+// @Override
+// public long getSettingsVersion() {
+// return SUDClickDetectorInfo.serialVersionUID;
+// }
+//
+// @Override
+// public Serializable getSettingsReference() {
+// return getSudClickDetectorInfo();
+// }
+
+ @Override
+ public boolean restoreSettings(PamControlledUnitSettings pamControlledUnitSettings) {
+ // have to leave this in since I've now run data with it using the ST settings,
+ // so some configs may return these !
+ Object o = pamControlledUnitSettings.getSettings();
+ if (o instanceof SUDClickDetectorInfo) {
+ sudClickDetectorInfo = (SUDClickDetectorInfo) o;
+ return true;
+ }
+ if (o instanceof ClickParameters) {
+ return super.restoreSettings(pamControlledUnitSettings);
+ }
+ return false;
+ }
+
+ /**
+ * @return the sudClickDetectorInfo
+ */
+ public SUDClickDetectorInfo getSudClickDetectorInfo() {
+ if (sudClickDetectorInfo == null) {
+ sudClickDetectorInfo = new SUDClickDetectorInfo();
+ }
+ return sudClickDetectorInfo;
+ }
+
+ /**
+ * @param sudClickDetectorInfo the sudClickDetectorInfo to set
+ */
+ public void setSudClickDetectorInfo(SUDClickDetectorInfo sudClickDetectorInfo) {
+ this.sudClickDetectorInfo = sudClickDetectorInfo;
+ }
+ /**
+ * Class to handle SoundTrap click detector settings without messing up
+ * the standard click detector ones which are needed for the classifier.
+ * @author dg50
+ *
+ */
+ private class SUDSettings implements PamSettings {
+
+ @Override
+ public String getUnitName() {
+ return STClickControl.this.getUnitName();
+ }
+
+ @Override
+ public String getUnitType() {
+ return STUNITTYPE;
+ }
+
+ @Override
+ public Serializable getSettingsReference() {
+ return sudClickDetectorInfo;
+ }
+
+ @Override
+ public long getSettingsVersion() {
+ return SUDClickDetectorInfo.serialVersionUID;
+ }
+
+ @Override
+ public boolean restoreSettings(PamControlledUnitSettings pamControlledUnitSettings) {
+ Object o = pamControlledUnitSettings.getSettings();
+ if (o instanceof SUDClickDetectorInfo) {
+ sudClickDetectorInfo = (SUDClickDetectorInfo) o;
+ return true;
+ }
+ return false;
+ }
+
+ }
}
diff --git a/src/soundtrap/STXMLFile.java b/src/soundtrap/STXMLFile.java
index a63a21f2..20ad30a9 100644
--- a/src/soundtrap/STXMLFile.java
+++ b/src/soundtrap/STXMLFile.java
@@ -12,6 +12,8 @@ import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
+import org.pamguard.x3.sud.SUDClickDetectorInfo;
+import org.pamguard.x3.sud.SUDXMLUtils;
import org.w3c.dom.Document;
import org.w3c.dom.NamedNodeMap;
import org.w3c.dom.Node;
@@ -41,6 +43,7 @@ public class STXMLFile {
private WAVInfo wavInfo;
private String soundTrapId;
private String dateFormat = defaultDateFormat;
+ private SUDClickDetectorInfo sudDetectorInfo;
public static void main(String[] args) {
@@ -70,6 +73,9 @@ public class STXMLFile {
}
private void unpackXMLDoc() throws Exception{
+ // try to get the new format complete settings as has been developed for the
+ // SUD direct reader.
+
cfgNodes = doc.getElementsByTagName("CFG");
NodeList procEvents = doc.getElementsByTagName("PROC_EVENT");
@@ -231,8 +237,14 @@ public class STXMLFile {
return null;
}
doc.getDocumentElement().normalize();
+
+ STXMLFile stXMLFile = new STXMLFile(doc, xmlFile, dateTimeFormat);
+
+ SUDXMLUtils sudXml = new SUDXMLUtils();
+ SUDClickDetectorInfo detectorInfo = sudXml.extractDetectorInfo(doc);
+ stXMLFile.setSudDetectorInfo(detectorInfo);
- return new STXMLFile(doc, xmlFile, dateTimeFormat);
+ return stXMLFile;
}
/**
@@ -299,4 +311,18 @@ public class STXMLFile {
}
return null;
}
+
+ /**
+ * @return the sudDetectorInfo
+ */
+ public SUDClickDetectorInfo getSudDetectorInfo() {
+ return sudDetectorInfo;
+ }
+
+ /**
+ * @param sudDetectorInfo the sudDetectorInfo to set
+ */
+ public void setSudDetectorInfo(SUDClickDetectorInfo sudDetectorInfo) {
+ this.sudDetectorInfo = sudDetectorInfo;
+ }
}
diff --git a/src/soundtrap/sud/SUDParamsDialog.java b/src/soundtrap/sud/SUDParamsDialog.java
new file mode 100644
index 00000000..335b20b9
--- /dev/null
+++ b/src/soundtrap/sud/SUDParamsDialog.java
@@ -0,0 +1,107 @@
+package soundtrap.sud;
+
+import java.awt.GridBagConstraints;
+import java.awt.GridBagLayout;
+import java.awt.Window;
+
+import javax.swing.JLabel;
+import javax.swing.JPanel;
+import javax.swing.JTextField;
+import javax.swing.border.TitledBorder;
+
+import org.pamguard.x3.sud.SUDClickDetectorInfo;
+
+import PamView.dialog.PamDialog;
+import PamView.dialog.PamGridBagContraints;
+import soundtrap.STClickControl;
+
+/**
+ * Display ST detection parameters. These cannot be changed so all boxes are disabled.
+ * @author dg50
+ *
+ */
+public class SUDParamsDialog extends PamDialog {
+
+ private STClickControl stClickControl;
+
+ private JTextField sampleRate, channels, threshold, blanking, preSamps, postSamps, len;
+
+ private static SUDParamsDialog singleInstance;
+
+ private SUDParamsDialog(Window parentFrame, STClickControl stClickControl) {
+ super(parentFrame, stClickControl.getUnitName(), false);
+ this.stClickControl = stClickControl;
+ JPanel mainPanel = new JPanel(new GridBagLayout());
+ mainPanel.setBorder(new TitledBorder("SoundTrap Detector Configuration"));
+ GridBagConstraints c = new PamGridBagContraints();
+ sampleRate = addThing(mainPanel, "Sample Rate", "Hz", c);
+ channels = addThing(mainPanel, "Num Channels", null, c);
+ threshold = addThing(mainPanel, "Threshold", "dB", c);
+ blanking = addThing(mainPanel, "Blanking", "samples", c);
+ preSamps = addThing(mainPanel, "Pre Samples", "samples", c);
+ postSamps = addThing(mainPanel, "Post Samples", "samples", c);
+ len = addThing(mainPanel, "Length", "samples", c);
+
+ setDialogComponent(mainPanel);
+ }
+
+ public static void showDialog(Window parent, STClickControl stClickControl) {
+// if (singleInstance == null) {
+ singleInstance = new SUDParamsDialog(parent, stClickControl);
+// }
+ singleInstance.setParams(stClickControl.getSudClickDetectorInfo());
+ singleInstance.setVisible(true);
+ return;
+ }
+
+ private void setParams(SUDClickDetectorInfo sudClickDetectorInfo) {
+ sampleRate.setText(String.format("%d", sudClickDetectorInfo.sampleRate));
+ channels.setText(String.format("%d", sudClickDetectorInfo.nChan));
+ threshold.setText(String.format("%3.1f", sudClickDetectorInfo.detThr));
+ blanking.setText(String.format("%d", sudClickDetectorInfo.blankingSamples));
+ preSamps.setText(String.format("%d", sudClickDetectorInfo.preSamples));
+ postSamps.setText(String.format("%d", sudClickDetectorInfo.postSamples));
+ len.setText(String.format("%d", sudClickDetectorInfo.lenSamples));
+ }
+
+ private JTextField addThing(JPanel mainPanel, String title, String postTit, GridBagConstraints c) {
+ c.gridx = 0;
+ mainPanel.add(new JLabel(title, JLabel.RIGHT), c);
+ c.gridx++;
+ STField field = new STField(5);
+ mainPanel.add(field, c);
+ if (postTit != null) {
+ c.gridx++;
+ mainPanel.add(new JLabel(postTit, JLabel.LEFT), c);
+ }
+ c.gridy++;
+ return field;
+ }
+
+ @Override
+ public boolean getParams() {
+ return true;
+ }
+
+ @Override
+ public void cancelButtonPressed() {
+ // TODO Auto-generated method stub
+
+ }
+
+ @Override
+ public void restoreDefaultSettings() {
+ // TODO Auto-generated method stub
+
+ }
+
+ private class STField extends JTextField {
+
+ public STField(int columns) {
+ super(columns);
+ setEditable(false);
+ }
+
+ }
+
+}
diff --git a/src/soundtrap/sud/SudFileDWVHandler.java b/src/soundtrap/sud/SudFileDWVHandler.java
index 11abd304..f03238a1 100644
--- a/src/soundtrap/sud/SudFileDWVHandler.java
+++ b/src/soundtrap/sud/SudFileDWVHandler.java
@@ -10,6 +10,7 @@ import java.util.List;
import org.pamguard.x3.sud.Chunk;
import org.pamguard.x3.sud.SudAudioInputStream;
import org.pamguard.x3.sud.SudDataInputStream;
+import org.pamguard.x3.sud.SudFileMap;
import Acquisition.AcquisitionControl;
import Acquisition.sud.SUDNotificationHandler;
@@ -73,7 +74,15 @@ public class SudFileDWVHandler implements SUDNotificationHandler {
// this is the wav sample rate, not the detector sample rate, so don't use it
// sampleRate = sudAudioInputStream.getFormat().getFrameRate();
// this is the right one
- sampleRate = sudAudioInputStream.getSudMap().clickDetSampleRate;
+ SudFileMap sudMap = sudAudioInputStream.getSudMap();
+ sampleRate = 0;
+ if (sudMap.detectorInfo != null) {
+ sampleRate = sudMap.detectorInfo.sampleRate;
+ stClickControl.setSudClickDetectorInfo(sudMap.detectorInfo);
+ }
+ if (sampleRate == 0) {
+ sampleRate = sudAudioInputStream.getSudMap().clickDetSampleRate;
+ }
fileStartMicros = sudAudioInputStream.getSudMap().getFirstChunkTimeMicros();
stClickControl.findRawDataBlock().setChannelMap(1);
stClickControl.findRawDataBlock().setSampleRate((float) sampleRate, true);
@@ -81,7 +90,7 @@ public class SudFileDWVHandler implements SUDNotificationHandler {
stClickControl.getSTAcquisition().acquisitionParameters.voltsPeak2Peak = STAcquisitionControl.SOUNDTRAPVP2P;
stClickControl.getSTAcquisition().getAcquisitionProcess().setSampleRate((float) sampleRate, true);
// System.out.printf("Open input stream fs = %3.1f\n", sampleRate);
-
+
}
@Override
@@ -104,10 +113,10 @@ public class SudFileDWVHandler implements SUDNotificationHandler {
String chunkName = "Unknown";
int chunkSize = sudChunk.buffer.length;
if (sudAudioInputStream.getChunkIDString(chunkID).equals("wav")) {
-
+
long millis = sudChunk.getChunkHeader().getMillisTime();
stClickControl.updateDisplayScrollers(millis);
-
+
if (sudAudioInputStream.isChunkIDWav(chunkID)) {
// chunkName = "RECORDINGS";
// System.out.printf("Chunk ID %d, size %d, type %s\n", chunkID, chunkSize,