Changes for page h. Test SFT1: System baseline test
Last modified by Rosa Van Tuijn on 2026/04/13 12:47
From version 1.17
edited by Tjalling Haije
on 2026/03/23 15:26
on 2026/03/23 15:26
Change comment:
There is no comment for this version
To version 2.4
edited by Rosa Van Tuijn
on 2026/03/24 16:16
on 2026/03/24 16:16
Change comment:
There is no comment for this version
Summary
-
Page properties (2 modified, 0 added, 0 removed)
Details
- Page properties
-
- Author
-
... ... @@ -1,1 +1,1 @@ 1 -XWiki. TjallingHaije1 +XWiki.RosaVanTuijn - Content
-
... ... @@ -10,7 +10,6 @@ 10 10 1. FR Health 11 11 1. FR and victim safety 12 12 13 - 14 14 **3. Hypotheses / Expectations (optional)** 15 15 See the claims from [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]], being: 16 16 ... ... @@ -22,7 +22,6 @@ 22 22 * CL06: improved FR health 23 23 * CL07: degraded mission efficiency 24 24 25 - 26 26 **4. Scenario / Context** 27 27 ASR3: Detailed indoor exploration 28 28 ... ... @@ -34,7 +34,6 @@ 34 34 ** **ANYmal **is mission-ready (battery, controls, and payload checked). 35 35 ** **C3I is online**, role-based views are configurated (Operator/ Analyst/ Team leader/ Safety officer); **5G pods** are installed for connection. 36 36 37 - 38 38 **5. Participants** 39 39 40 40 * FR team ROBOT: ... ... @@ -107,12 +107,78 @@ 107 107 * FR Team ROBOT does Building B 108 108 * FR team HUMAN does Building A 109 109 110 - 111 111 **9. Measurements** 112 112 113 -See the measurements from [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]] 109 +See the measurements from [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]]. Compare outcomes between the two conditions teams if applicable. 114 114 111 +|**Claim: Safety**|**Measurment(s)**|**How to test ** 112 +|Safer decision before entry|Safe path chosen|((( 113 +* Define ground-truth safe path(s) beforehand so that they can be compared to chosen entry path during the trial 114 +* Score paths beforehand: safe/ suboptimal/ unsafe 115 +))) 116 +|Better decision overall|"How good were you decisions looking back?"|((( 117 +* Hotwash reflection question (likert + open) 118 +* Observations of errors (e.g. near-incidents, unsafe choices) 119 +))) 120 +|Fewer dangerous situations|Near-incident reports|((( 121 +* Define hazards in buildings beforehand and check (with observations of during hotwash) if these hazards are avoided or not 122 +))) 115 115 124 +|**Claim: Efficiency**|**Measurment(s)**|**How to test ** 125 +|Faster mission execution|Total mission completion time|((( 126 +* Timestamp of start (after briefing) till all located victims have been extracted 127 +))) 128 +|Faster inside performance|First responder time inside|((( 129 +* Timestamp of entry building till exist building (last located victim is exstracted) 130 +))) 131 + 132 +|**Claim: Situation Awareness**|**Measurment(s)**|**How to test ** 133 +|Before entry |((( 134 +Correctness of reported hazards, victims, layout 135 +)))|((( 136 +* Before entry let squad leader and entry team (seperatly) sketch/map or describe inside of building (harzards, victims, layout) > compare these with the ground-truth 137 + 138 +((( 139 + 140 +))) 141 +))) 142 +|During entry|((( 143 +FR path logs 144 +)))|((( 145 +* Compare plannend vs. executed path (how many or how big were the deviations? and were they acceptable?) and identify detours caused by misjudgment or new information 146 + 147 +((( 148 + 149 +))) 150 +))) 151 +|After mission|((( 152 +Correctness of reported hazards, victims, layout 153 +)))|((( 154 +* Repeat sketching/mapping or descriptive task and compare the outcomes (improvement/ deterioration vs pre-entry) 155 +))) 156 + 157 +|**Claim: Health**|**Measurment(s)**|**How to test ** 158 +|((( 159 +Health remains acceptable 160 +)))|((( 161 + 162 + 163 +Health indicators 164 +)))|((( 165 +* Define acceptable ranges (e.g. HR, temperature, etc.) beforehand and log these parapeter ranges 166 +))) 167 +|((( 168 +Issues handled in time 169 +)))|((( 170 + 171 + 172 +Health issues tackled in acceptable time 173 +)))|((( 174 +* Log when 175 +))) 176 + 177 + 178 + 116 116 **10. Procedure (Step-by-Step)** 117 117 118 118 1. __Team preparation__: ... ... @@ -134,32 +134,33 @@ 134 134 111. Time for tackling any health alerts 135 135 111. Track FR path 136 136 1. __Hotwash__: 137 -11. Questions on: 200 +11. Questions (specified per role, couple questions per topic) on: 138 138 111. Near-incident report 139 139 111. Path inside 140 140 111. FR location and status of hazards, victims, and layout 141 -111. Mission effectiveness 204 +111. Mission effectiveness: e.g. self reported effectiveness of decisions 142 142 111. Self-reported workload 143 143 111. Trust survey 207 +11. Group discussion with feedback 144 144 145 145 **~11. Planning:** 146 146 147 147 * Team preparation: Day before. 1 hour. 148 148 * Trial 1: 149 -** Setup usecase: Test day 1. 09:00 - 09:30213 +** Setup scenario: Test day 1. 09:00 - 09:30 150 150 ** Setup tech and perform checks: Test day 1. 09:00 - 09:30 151 151 ** Briefing of each team individually (outside building): Test day 1: 09:00 - 09:30 152 152 ** Mission execution: Test day 1. 09:30 - 10:30 153 153 ** Hot wash: Test day 1. 10:30-11:00 154 -** Usecase reset: test day 1. 10:30-11:00218 +** Scenario reset: test day 1. 10:30-11:00 155 155 * Teams switch location: Test day 1. 11:00-11:15 156 156 * Trial 2: 157 -** Setup usecase: Test day 1. 12:30 - 13:00221 +** (Setup scenario: Test day 1. 12:30 - 13:00) 158 158 ** Setup tech and perform checks: Test day 1. 12:30 - 13:00 159 159 ** Briefing of each team individually (outside building): Test day 1: 12:30 - 13:00 160 160 ** Mission execution: Test day 1. 13:00 - 14:00 161 161 ** Hot wash: Test day 1. 14:00-14:30 162 -* *Cleanup area and store tech. 14:00-14:30226 +* Cleanup area and store tech. 14:00-14:30 163 163 164 164 165 165 ... ... @@ -178,7 +178,6 @@ 178 178 ** ~~8x Actors or dummies for victims. 179 179 * 8x Clipboards with papers and pens for hotwash and questionnaires 180 180 181 - 182 182 **13. Dependencies** 183 183 184 184 * Availability of materials ... ... @@ -189,7 +189,6 @@ 189 189 ** People willing to play victim 190 190 ** Experiment leaders 191 191 192 - 193 193 **14. Success Criteria** 194 194 When mission(s) completed by both teams, and we can compare their performance using the metrics. 195 195