Skip to Content
Version 4.1 by Rosa Van Tuijn on 2026/03/25 14:36
Hide last authors
Tjalling Haije 1.6 1 **Experiment title**
Tjalling Haije 1.16 2 System baseline test
Tjalling Haije 1.2 3
Tjalling Haije 1.6 4
Tjalling Haije 1.2 5 **2. Objective / Research Question**
Tjalling Haije 1.14 6 Do the SYNERGISE technologies together improve the SYNERGISE [[Objectives>>2\. Specification.Objectives.WebHome]] (stated below) without negative effects?
Tjalling Haije 1.2 7
Tjalling Haije 1.6 8 1. Mission effectiveness
9 1. Mission efficiency
10 1. FR Health
11 1. FR and victim safety
12
Tjalling Haije 1.2 13 **3. Hypotheses / Expectations (optional)**
Tjalling Haije 1.6 14 See the claims from [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]], being:
Tjalling Haije 1.2 15
Tjalling Haije 1.6 16 * CL01: Improve safety for responders
17 * CL02: Increased SA
18 * CL03: Improved mission effectiveness
19 * CL04: acceptable workload
20 * CL05: trustworthy SA
21 * CL06: improved FR health
22 * CL07: degraded mission efficiency
23
Tjalling Haije 1.2 24 **4. Scenario / Context**
Tjalling Haije 1.7 25 ASR3: Detailed indoor exploration
Tjalling Haije 1.2 26
Tjalling Haije 1.7 27 Preconditions from [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]] :
28
29 * Structure is flagged as (relatively?) **safe for direct human entry**; sector/ worksite allocated.
30 * **FRs wearables** distributed, connected, and calibrated.
31 * Specific to scenario with ANYmal:
32 ** **ANYmal **is mission-ready (battery, controls, and payload checked).
33 ** **C3I is online**, role-based views are configurated (Operator/ Analyst/ Team leader/ Safety officer); **5G pods** are installed for connection.
34
Tjalling Haije 1.2 35 **5. Participants**
36
Tjalling Haije 1.7 37 * FR team ROBOT:
Tjalling Haije 1.3 38 ** 1x Squad leaders
39 ** 1x Robot operator
40 ** 1x Robot analyst
41 ** 2x First responder
Tjalling Haije 1.7 42 * FR team HUMAN:
Tjalling Haije 1.3 43 ** 1x Squad leaders
44 ** 2x First reponder
45 * Base of Operations:
Tjalling Haije 1.6 46 ** 1x Safety officer / medical specialist
Tjalling Haije 1.3 47 ** Team lead
Tjalling Haije 1.2 48
Tjalling Haije 1.6 49 Number of participants: 9 
50
Tjalling Haije 1.5 51
Tjalling Haije 1.2 52 **6. System(s) Under Test**
53
Tjalling Haije 1.7 54 |(% style="width:267px" %)Technology|(% style="width:1250px" %)Description
55 |(% style="width:267px" %)ANYmal|(% style="width:1250px" %)(((
56 * Robust mobility over uneven terrain (stairs, debris, slopes).
57 * Multi-sensor payload (video, LIDAR, thermal??)
58 * Reliable wireless communication/ connection in degraded environments.
59 * Support for tele‑operation with clear operator feedback.
60 * Link with C3I, such as adding victims as pin on map.
61 * NO autonomy.
62 )))
63 |(% style="width:267px" %)C3I (BoO and tablet version)|(% style="width:1250px" %)(((
64 * Interoperability with health and location sensors.
65 * Robust operation under limited connectivity (sync when possible).
66 * Usable tablet interface for field conditions (gloves, stress, time pressure).
67 )))
68 |(% style="width:267px" %)Health sensors|(% style="width:1250px" %)(((
69 * Accurate and reliable physiological sensing (e.g. heart rate, exertion proxies).
70 * Comfortable, non‑intrusive form factor suitable for long missions.
71 * Secure wireless data transmission to C3I systems.
72 * Real‑time alerting and trend monitoring.
73 )))
74 |(% style="width:267px" %)Location sensors|(% style="width:1250px" %)(((
75 * Indoor and outdoor localisation capability.
76 * Sufficient accuracy and update rate for safety‑critical use.
77 * Integration with C3I and mapping systems.
78 * Robustness to environmental interference (dust, debris, structures, height differences).
79 )))
Tjalling Haije 1.2 80
Tjalling Haije 1.7 81
82
Tjalling Haije 1.2 83 **7. Tasks**
84
Tjalling Haije 1.6 85 [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]]
86
87 [[UC04.4: Detailed indoor exploration without ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYmal (USAR).WebHome]] 
88
89
Tjalling Haije 1.2 90 **8. Experimental Conditions / Variants**
Tjalling Haije 1.7 91
92 Two buildings:
93
Tjalling Haije 1.8 94 * Building A: Easier location, obstacles and environmental conditions. (clear visibility, ..)
95 * Building B: Challenging location, obstacles and environmental conditions for humans and robots (e.g. smoke and obstacles).
Tjalling Haije 1.7 96
Tjalling Haije 1.8 97 Trial 1:
Tjalling Haije 1.7 98
Tjalling Haije 1.8 99 * FR Team ROBOT does Building A
100 * FR team HUMAN does Building B
Tjalling Haije 1.7 101
Tjalling Haije 1.8 102 Trial 2:
Tjalling Haije 1.7 103
Tjalling Haije 1.8 104 * FR Team ROBOT does Building B
105 * FR team HUMAN does Building A
Tjalling Haije 1.7 106
Tjalling Haije 1.2 107 **9. Measurements**
108
Rosa Van Tuijn 2.2 109 See the measurements from [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]]. Compare outcomes between the two conditions teams if applicable.
Tjalling Haije 1.2 110
Rosa Van Tuijn 3.1 111 |**Claim: Safety [CL01]**|**Measurment(s)**|**How to test **
Rosa Van Tuijn 2.1 112 |Safer decision before entry|Safe path chosen|(((
113 * Define ground-truth safe path(s) beforehand so that they can be compared to chosen entry path during the trial
114 * Score paths beforehand: safe/ suboptimal/ unsafe
115 )))
116 |Better decision overall|"How good were you decisions looking back?"|(((
117 * Hotwash reflection question (likert + open)
Rosa Van Tuijn 2.2 118 * Observations of errors (e.g. near-incidents, unsafe choices)
Rosa Van Tuijn 2.1 119 )))
Rosa Van Tuijn 2.2 120 |Fewer dangerous situations|Near-incident reports|(((
121 * Define hazards in buildings beforehand and check (with observations of during hotwash) if these hazards are avoided or not
122 )))
Tjalling Haije 1.6 123
Rosa Van Tuijn 3.1 124 |**Claim: Situation Awareness [CL02]**|**Measurment(s)**|**How to test **
Rosa Van Tuijn 2.2 125 |Before entry |(((
126 Correctness of reported hazards, victims, layout
127 )))|(((
Rosa Van Tuijn 2.3 128 * Before entry let squad leader and entry team (seperatly) sketch/map or describe inside of building (harzards, victims, layout) > compare these with the ground-truth
129
130 (((
Rosa Van Tuijn 2.2 131
132 )))
Rosa Van Tuijn 2.3 133 )))
Rosa Van Tuijn 2.2 134 |During entry|(((
135 FR path logs
136 )))|(((
Rosa Van Tuijn 2.3 137 * Compare plannend vs. executed path (how many or how big were the deviations? and were they acceptable?) and identify detours caused by misjudgment or new information
138
139 (((
Rosa Van Tuijn 2.2 140
141 )))
Rosa Van Tuijn 2.3 142 )))
Rosa Van Tuijn 2.2 143 |After mission|(((
144 Correctness of reported hazards, victims, layout
Rosa Van Tuijn 2.3 145 )))|(((
146 * Repeat sketching/mapping or descriptive task and compare the outcomes (improvement/ deterioration vs pre-entry)
147 )))
148
Rosa Van Tuijn 3.2 149 |**Claim: Mission effectiveness [CL03] [CL04] [CL05]**|**Measurment(s)**|**How to test **
Rosa Van Tuijn 2.4 150 |(((
Rosa Van Tuijn 3.1 151 Acceptable workload
152 )))|(((
Rosa Van Tuijn 3.2 153
Rosa Van Tuijn 3.1 154
155 NASA-TLX
156 )))|(((
157 * Shortened post-trial NASA-TLX questionnaire and analyze score (discuss if these with experts to check if these are "acceptable")
158
159 (((
160
161 )))
162 )))
163 |(((
164 Appropriate trust
165 )))|(((
Rosa Van Tuijn 3.2 166
Rosa Van Tuijn 3.1 167
168 Trust survey / interviews
169 )))|(((
170 *
171 )))
172 |(((
173 Decision confidence
174 )))|(((
Rosa Van Tuijn 3.2 175
Rosa Van Tuijn 3.1 176
177 Retrospective decision quality rating
178 )))|
179
180 |**Claim: Health [CL06]**|**Measurment(s)**|**How to test **
181 |(((
Rosa Van Tuijn 2.4 182 Health remains acceptable
Rosa Van Tuijn 2.3 183 )))|(((
Rosa Van Tuijn 3.2 184
Rosa Van Tuijn 2.4 185
186 Health indicators
187 )))|(((
Rosa Van Tuijn 3.1 188 * Define acceptable ranges (e.g. HR, temperature, etc.) beforehand and log these ranges
Rosa Van Tuijn 2.3 189 )))
Rosa Van Tuijn 2.4 190 |(((
191 Issues handled in time
Rosa Van Tuijn 2.3 192 )))|(((
Rosa Van Tuijn 3.2 193
Rosa Van Tuijn 2.4 194
195 Health issues tackled in acceptable time
196 )))|(((
Rosa Van Tuijn 3.1 197 * Timestamp when acceptable ranges are breached till moment detection and intervention moment (compare this with predefined response thresholds)
Rosa Van Tuijn 2.3 198 )))
Rosa Van Tuijn 2.1 199
Rosa Van Tuijn 3.1 200 |**Claim: Mission Efficiency [CL07]**|**Measurment(s)**|**How to test **
201 |Faster mission execution|Total mission completion time|(((
202 * Timestamp of start (after briefing) till all located victims have been extracted
203 )))
204 |Faster inside performance|First responder time inside|(((
205 * Timestamp of entry building till exist building (last located victim is exstracted)
206 )))
Rosa Van Tuijn 2.2 207
Tjalling Haije 1.2 208 **10. Procedure (Step-by-Step)**
209
Tjalling Haije 1.15 210 1. __Team preparation__:
Tjalling Haije 1.8 211 11. FR team ROBOT is trained in operating the robot, analysing the camera stream, and pinning information on C3I
212 11. Squad leader and BoO are trained for both teams in interpreting the health and location sensor data.
213 11. Experiment leader knows where hazards and victims are in both buildings.
Tjalling Haije 1.13 214 11. Anymal robot is tested with network in target buildings for experiment (without FR themselves seeing building!)
Tjalling Haije 1.15 215 1. __Experiment preparation:__
Tjalling Haije 1.11 216 11. Building is prepped with victims, hazards, etc.
Tjalling Haije 1.15 217 1. __Briefing:__
Tjalling Haije 1.8 218 11. Same for both teams for both building types: Building has been inspected from outside and is estimated safe for entry. Situation inside unknown. Victims possibly present. Goal: find, assess, and extract victims while avoiding and mapping hazards.
219 11. One person is notified that their health sensor will be triggered during the Building B (difficult building) situation.
Tjalling Haije 1.15 220 1. __Execution:__
Tjalling Haije 1.8 221 11. For FR team ROBOT, see: [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]]
222 11. For FR team HUMAN, see: [[UC04.4: Detailed indoor exploration without ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYmal (USAR).WebHome]]
223 11. During execution of both teams, performance will be measured by experiment leader:
224 111. Mission completion time
225 111. Health indicators stay within acceptable range
226 111. Time for tackling any health alerts
227 111. Track FR path
Tjalling Haije 1.15 228 1. __Hotwash__:
Tjalling Haije 1.21 229 11. Questions (specified per role, couple questions per topic) on:
Tjalling Haije 1.8 230 111. Near-incident report
231 111. Path inside
232 111. FR location and status of hazards, victims, and layout
Tjalling Haije 1.25 233 111. Mission effectiveness: e.g. self reported effectiveness of decisions
Tjalling Haije 1.8 234 111. Self-reported workload
235 111. Trust survey
Tjalling Haije 1.18 236 11. Group discussion with feedback
Tjalling Haije 1.2 237
Tjalling Haije 1.15 238 **~11. Planning:**
Tjalling Haije 1.6 239
Tjalling Haije 1.13 240 * Team preparation: Day before. 1 hour.
Tjalling Haije 1.11 241 * Trial 1:
Tjalling Haije 1.22 242 ** Setup scenario: Test day 1. 09:00 - 09:30
Tjalling Haije 1.11 243 ** Setup tech and perform checks: Test day 1. 09:00 - 09:30
244 ** Briefing of each team individually (outside building): Test day 1: 09:00 - 09:30
245 ** Mission execution: Test day 1. 09:30 - 10:30
246 ** Hot wash: Test day 1. 10:30-11:00
Tjalling Haije 1.22 247 ** Scenario reset: test day 1. 10:30-11:00
Tjalling Haije 1.11 248 * Teams switch location: Test day 1. 11:00-11:15
249 * Trial 2: 
Tjalling Haije 1.23 250 ** (Setup scenario: Test day 1. 12:30 - 13:00)
Tjalling Haije 1.11 251 ** Setup tech and perform checks: Test day 1. 12:30 - 13:00
252 ** Briefing of each team individually (outside building): Test day 1: 12:30 - 13:00
253 ** Mission execution: Test day 1. 13:00 - 14:00
254 ** Hot wash: Test day 1. 14:00-14:30
Tjalling Haije 1.23 255 * Cleanup area and store tech. 14:00-14:30
Tjalling Haije 1.2 256
Tjalling Haije 1.6 257
Tjalling Haije 1.2 258
Tjalling Haije 1.15 259 **12. Materials**
Tjalling Haije 1.2 260
Tjalling Haije 1.12 261 * For FRs:
262 ** 1x Anymal with teleop setup and networking
263 ** 2x Tablet with C3I for squad leader (one spare)
264 ** 1x Base of Operations with computer with C3I
265 ** 4x Health sensors
266 ** 4x Location sensors
267 ** 8x walky talky with two channels (?) (for FR team and BoO).
268 * For usecase:
269 ** 2x training building (~~100m2?)
270 ** Enviromental challenges: smoke machine, obstacles, etc.
271 ** ~~8x Actors or dummies for victims.
272 * 8x Clipboards with papers and pens for hotwash and questionnaires
Tjalling Haije 1.6 273
Tjalling Haije 1.15 274 **13. Dependencies**
Tjalling Haije 1.2 275
Tjalling Haije 1.13 276 * Availability of materials
Tjalling Haije 1.15 277 * Availability and functioning of ANYmal robot with teleoperation and network in target buildings
Tjalling Haije 1.13 278 * Availability of functioning network between robot, FRs, and BoO.
279 * Availability of:
280 ** Experiment participants
281 ** People willing to play victim
282 ** Experiment leaders
Tjalling Haije 1.6 283
Tjalling Haije 1.15 284 **14. Success Criteria**
285 When mission(s) completed by both teams, and we can compare their performance using the metrics.
Tjalling Haije 1.2 286
287
288