Skip to Content
Last modified by Rosa Van Tuijn on 2026/04/13 12:47
From version 2.4
edited by Rosa Van Tuijn
on 2026/03/24 16:16
Change comment: There is no comment for this version
To version 1.15
edited by Tjalling Haije
on 2026/03/23 15:25
Change comment: There is no comment for this version

Summary

Details

Page properties
Title
... ... @@ -1,1 +1,1 @@
1 -h. Test SFT1: System baseline test
1 +h. Test SFT1
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.RosaVanTuijn
1 +XWiki.TjallingHaije
Content
... ... @@ -1,5 +1,5 @@
1 1  **Experiment title**
2 -System baseline test
2 +[Short, descriptive title]
3 3  
4 4  
5 5  **2. Objective / Research Question**
... ... @@ -10,6 +10,7 @@
10 10  1. FR Health
11 11  1. FR and victim safety
12 12  
13 +
13 13  **3. Hypotheses / Expectations (optional)**
14 14  See the claims from [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]], being:
15 15  
... ... @@ -21,6 +21,7 @@
21 21  * CL06: improved FR health
22 22  * CL07: degraded mission efficiency
23 23  
25 +
24 24  **4. Scenario / Context**
25 25  ASR3: Detailed indoor exploration
26 26  
... ... @@ -32,6 +32,7 @@
32 32  ** **ANYmal **is mission-ready (battery, controls, and payload checked).
33 33  ** **C3I is online**, role-based views are configurated (Operator/ Analyst/ Team leader/ Safety officer); **5G pods** are installed for connection.
34 34  
37 +
35 35  **5. Participants**
36 36  
37 37  * FR team ROBOT:
... ... @@ -104,78 +104,12 @@
104 104  * FR Team ROBOT does Building B
105 105  * FR team HUMAN does Building A
106 106  
110 +
107 107  **9. Measurements**
108 108  
109 -See the measurements from [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]]. Compare outcomes between the two conditions teams if applicable.
113 +See the measurements from [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]]
110 110  
111 -|**Claim: Safety**|**Measurment(s)**|**How to test **
112 -|Safer decision before entry|Safe path chosen|(((
113 -* Define ground-truth safe path(s) beforehand so that they can be compared to chosen entry path during the trial
114 -* Score paths beforehand: safe/ suboptimal/ unsafe
115 -)))
116 -|Better decision overall|"How good were you decisions looking back?"|(((
117 -* Hotwash reflection question (likert + open)
118 -* Observations of errors (e.g. near-incidents, unsafe choices)
119 -)))
120 -|Fewer dangerous situations|Near-incident reports|(((
121 -* Define hazards in buildings beforehand and check (with observations of during hotwash) if these hazards are avoided or not
122 -)))
123 123  
124 -|**Claim: Efficiency**|**Measurment(s)**|**How to test **
125 -|Faster mission execution|Total mission completion time|(((
126 -* Timestamp of start (after briefing) till all located victims have been extracted
127 -)))
128 -|Faster inside performance|First responder time inside|(((
129 -* Timestamp of entry building till exist building (last located victim is exstracted)
130 -)))
131 -
132 -|**Claim: Situation Awareness**|**Measurment(s)**|**How to test **
133 -|Before entry |(((
134 -Correctness of reported hazards, victims, layout
135 -)))|(((
136 -* Before entry let squad leader and entry team (seperatly) sketch/map or describe inside of building (harzards, victims, layout) > compare these with the ground-truth
137 -
138 -(((
139 -
140 -)))
141 -)))
142 -|During entry|(((
143 -FR path logs
144 -)))|(((
145 -* Compare plannend vs. executed path (how many or how big were the deviations? and were they acceptable?) and identify detours caused by misjudgment or new information
146 -
147 -(((
148 -
149 -)))
150 -)))
151 -|After mission|(((
152 -Correctness of reported hazards, victims, layout
153 -)))|(((
154 -* Repeat sketching/mapping or descriptive task and compare the outcomes (improvement/ deterioration vs pre-entry)
155 -)))
156 -
157 -|**Claim: Health**|**Measurment(s)**|**How to test **
158 -|(((
159 -Health remains acceptable
160 -)))|(((
161 -
162 -
163 -Health indicators
164 -)))|(((
165 -* Define acceptable ranges (e.g. HR, temperature, etc.) beforehand and log these parapeter ranges
166 -)))
167 -|(((
168 -Issues handled in time
169 -)))|(((
170 -
171 -
172 -Health issues tackled in acceptable time
173 -)))|(((
174 -* Log when
175 -)))
176 -
177 -
178 -
179 179  **10. Procedure (Step-by-Step)**
180 180  
181 181  1. __Team preparation__:
... ... @@ -197,33 +197,32 @@
197 197  111. Time for tackling any health alerts
198 198  111. Track FR path
199 199  1. __Hotwash__:
200 -11. Questions (specified per role, couple questions per topic) on:
137 +11. Questions on:
201 201  111. Near-incident report
202 202  111. Path inside
203 203  111. FR location and status of hazards, victims, and layout
204 -111. Mission effectiveness: e.g. self reported effectiveness of decisions
141 +111. Mission effectiveness
205 205  111. Self-reported workload
206 206  111. Trust survey
207 -11. Group discussion with feedback
208 208  
209 209  **~11. Planning:**
210 210  
211 211  * Team preparation: Day before. 1 hour.
212 212  * Trial 1:
213 -** Setup scenario: Test day 1. 09:00 - 09:30
149 +** Setup usecase: Test day 1. 09:00 - 09:30
214 214  ** Setup tech and perform checks: Test day 1. 09:00 - 09:30
215 215  ** Briefing of each team individually (outside building): Test day 1: 09:00 - 09:30
216 216  ** Mission execution: Test day 1. 09:30 - 10:30
217 217  ** Hot wash: Test day 1. 10:30-11:00
218 -** Scenario reset: test day 1. 10:30-11:00
154 +** Usecase reset: test day 1. 10:30-11:00
219 219  * Teams switch location: Test day 1. 11:00-11:15
220 220  * Trial 2: 
221 -** (Setup scenario: Test day 1. 12:30 - 13:00)
157 +** Setup usecase: Test day 1. 12:30 - 13:00
222 222  ** Setup tech and perform checks: Test day 1. 12:30 - 13:00
223 223  ** Briefing of each team individually (outside building): Test day 1: 12:30 - 13:00
224 224  ** Mission execution: Test day 1. 13:00 - 14:00
225 225  ** Hot wash: Test day 1. 14:00-14:30
226 -* Cleanup area and store tech. 14:00-14:30
162 +** Cleanup area and store tech. 14:00-14:30
227 227  
228 228  
229 229  
... ... @@ -242,6 +242,7 @@
242 242  ** ~~8x Actors or dummies for victims.
243 243  * 8x Clipboards with papers and pens for hotwash and questionnaires
244 244  
181 +
245 245  **13. Dependencies**
246 246  
247 247  * Availability of materials
... ... @@ -252,6 +252,7 @@
252 252  ** People willing to play victim
253 253  ** Experiment leaders
254 254  
192 +
255 255  **14. Success Criteria**
256 256  When mission(s) completed by both teams, and we can compare their performance using the metrics.
257 257