Skip to Content
Version 3.3 by Rosa Van Tuijn on 2026/03/25 14:36
Show last authors
1 **Experiment title**
2 System baseline test
3
4
5 **2. Objective / Research Question**
6 Do the SYNERGISE technologies together improve the SYNERGISE [[Objectives>>2\. Specification.Objectives.WebHome]] (stated below) without negative effects?
7
8 1. Mission effectiveness
9 1. Mission efficiency
10 1. FR Health
11 1. FR and victim safety
12
13 **3. Hypotheses / Expectations (optional)**
14 See the claims from [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]], being:
15
16 * CL01: Improve safety for responders
17 * CL02: Increased SA
18 * CL03: Improved mission effectiveness
19 * CL04: acceptable workload
20 * CL05: trustworthy SA
21 * CL06: improved FR health
22 * CL07: degraded mission efficiency
23
24 **4. Scenario / Context**
25 ASR3: Detailed indoor exploration
26
27 Preconditions from [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]] :
28
29 * Structure is flagged as (relatively?) **safe for direct human entry**; sector/ worksite allocated.
30 * **FRs wearables** distributed, connected, and calibrated.
31 * Specific to scenario with ANYmal:
32 ** **ANYmal **is mission-ready (battery, controls, and payload checked).
33 ** **C3I is online**, role-based views are configurated (Operator/ Analyst/ Team leader/ Safety officer); **5G pods** are installed for connection.
34
35 **5. Participants**
36
37 * FR team ROBOT:
38 ** 1x Squad leaders
39 ** 1x Robot operator
40 ** 1x Robot analyst
41 ** 2x First responder
42 * FR team HUMAN:
43 ** 1x Squad leaders
44 ** 2x First reponder
45 * Base of Operations:
46 ** 1x Safety officer / medical specialist
47 ** Team lead
48
49 Number of participants: 9 
50
51
52 **6. System(s) Under Test**
53
54 |(% style="width:267px" %)Technology|(% style="width:1250px" %)Description
55 |(% style="width:267px" %)ANYmal|(% style="width:1250px" %)(((
56 * Robust mobility over uneven terrain (stairs, debris, slopes).
57 * Multi-sensor payload (video, LIDAR, thermal??)
58 * Reliable wireless communication/ connection in degraded environments.
59 * Support for tele‑operation with clear operator feedback.
60 * Link with C3I, such as adding victims as pin on map.
61 * NO autonomy.
62 )))
63 |(% style="width:267px" %)C3I (BoO and tablet version)|(% style="width:1250px" %)(((
64 * Interoperability with health and location sensors.
65 * Robust operation under limited connectivity (sync when possible).
66 * Usable tablet interface for field conditions (gloves, stress, time pressure).
67 )))
68 |(% style="width:267px" %)Health sensors|(% style="width:1250px" %)(((
69 * Accurate and reliable physiological sensing (e.g. heart rate, exertion proxies).
70 * Comfortable, non‑intrusive form factor suitable for long missions.
71 * Secure wireless data transmission to C3I systems.
72 * Real‑time alerting and trend monitoring.
73 )))
74 |(% style="width:267px" %)Location sensors|(% style="width:1250px" %)(((
75 * Indoor and outdoor localisation capability.
76 * Sufficient accuracy and update rate for safety‑critical use.
77 * Integration with C3I and mapping systems.
78 * Robustness to environmental interference (dust, debris, structures, height differences).
79 )))
80
81
82
83 **7. Tasks**
84
85 [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]]
86
87 [[UC04.4: Detailed indoor exploration without ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYmal (USAR).WebHome]] 
88
89
90 **8. Experimental Conditions / Variants**
91
92 Two buildings:
93
94 * Building A: Easier location, obstacles and environmental conditions. (clear visibility, ..)
95 * Building B: Challenging location, obstacles and environmental conditions for humans and robots (e.g. smoke and obstacles).
96
97 Trial 1:
98
99 * FR Team ROBOT does Building A
100 * FR team HUMAN does Building B
101
102 Trial 2:
103
104 * FR Team ROBOT does Building B
105 * FR team HUMAN does Building A
106
107 **9. Measurements**
108
109 See the measurements from [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]]. Compare outcomes between the two conditions teams if applicable.
110
111 |**Claim: Safety [CL01]**|**Measurment(s)**|**How to test **
112 |Safer decision before entry|Safe path chosen|(((
113 * Define ground-truth safe path(s) beforehand so that they can be compared to chosen entry path during the trial
114 * Score paths beforehand: safe/ suboptimal/ unsafe
115 )))
116 |Better decision overall|"How good were you decisions looking back?"|(((
117 * Hotwash reflection question (likert + open)
118 * Observations of errors (e.g. near-incidents, unsafe choices)
119 )))
120 |Fewer dangerous situations|Near-incident reports|(((
121 * Define hazards in buildings beforehand and check (with observations of during hotwash) if these hazards are avoided or not
122 )))
123
124 |**Claim: Situation Awareness [CL02]**|**Measurment(s)**|**How to test **
125 |Before entry |(((
126 Correctness of reported hazards, victims, layout
127 )))|(((
128 * Before entry let squad leader and entry team (seperatly) sketch/map or describe inside of building (harzards, victims, layout) > compare these with the ground-truth
129
130 (((
131
132 )))
133 )))
134 |During entry|(((
135 FR path logs
136 )))|(((
137 * Compare plannend vs. executed path (how many or how big were the deviations? and were they acceptable?) and identify detours caused by misjudgment or new information
138
139 (((
140
141 )))
142 )))
143 |After mission|(((
144 Correctness of reported hazards, victims, layout
145 )))|(((
146 * Repeat sketching/mapping or descriptive task and compare the outcomes (improvement/ deterioration vs pre-entry)
147 )))
148
149 |**Claim: Mission effectiveness [CL03] [CL04] [CL05]**|**Measurment(s)**|**How to test **
150 |(((
151 Acceptable workload
152 )))|(((
153
154
155 NASA-TLX
156 )))|(((
157 * Shortened post-trial NASA-TLX questionnaire and analyze score (discuss if these with experts to check if these are "acceptable")
158
159 (((
160
161 )))
162 )))
163 |(((
164 Appropriate trust
165 )))|(((
166
167
168 Trust survey / interviews
169 )))|(((
170 *
171 )))
172 |(((
173 Decision confidence
174 )))|(((
175
176
177 Retrospective decision quality rating
178 )))|
179
180 |**Claim: Health [CL06]**|**Measurment(s)**|**How to test **
181 |(((
182 Health remains acceptable
183 )))|(((
184
185
186 Health indicators
187 )))|(((
188 * Define acceptable ranges (e.g. HR, temperature, etc.) beforehand and log these ranges
189 )))
190 |(((
191 Issues handled in time
192 )))|(((
193
194
195 Health issues tackled in acceptable time
196 )))|(((
197 * Timestamp when acceptable ranges are breached till moment detection and intervention moment (compare this with predefined response thresholds)
198 )))
199
200 |**Claim: Mission Efficiency [CL07]**|**Measurment(s)**|**How to test **
201 |Faster mission execution|Total mission completion time|(((
202 * Timestamp of start (after briefing) till all located victims have been extracted
203 )))
204 |Faster inside performance|First responder time inside|(((
205 * Timestamp of entry building till exist building (last located victim is exstracted)
206 )))
207
208 **10. Procedure (Step-by-Step)**
209
210 1. __Team preparation__:
211 11. FR team ROBOT is trained in operating the robot, analysing the camera stream, and pinning information on C3I
212 11. Squad leader and BoO are trained for both teams in interpreting the health and location sensor data.
213 11. Experiment leader knows where hazards and victims are in both buildings.
214 11. Anymal robot is tested with network in target buildings for experiment (without FR themselves seeing building!)
215 1. __Experiment preparation:__
216 11. Building is prepped with victims, hazards, etc.
217 1. __Briefing:__
218 11. Same for both teams for both building types: Building has been inspected from outside and is estimated safe for entry. Situation inside unknown. Victims possibly present. Goal: find, assess, and extract victims while avoiding and mapping hazards.
219 11. One person is notified that their health sensor will be triggered during the Building B (difficult building) situation.
220 1. __Execution:__
221 11. For FR team ROBOT, see: [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]]
222 11. For FR team HUMAN, see: [[UC04.4: Detailed indoor exploration without ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYmal (USAR).WebHome]]
223 11. During execution of both teams, performance will be measured by experiment leader:
224 111. Mission completion time
225 111. Health indicators stay within acceptable range
226 111. Time for tackling any health alerts
227 111. Track FR path
228 1. __Hotwash__:
229 11. Questions (specified per role, couple questions per topic) on:
230 111. Near-incident report
231 111. Path inside
232 111. FR location and status of hazards, victims, and layout
233 111. Mission effectiveness: e.g. self reported effectiveness of decisions
234 111. Self-reported workload
235 111. Trust survey
236 11. Group discussion with feedback
237
238 **~11. Planning:**
239
240 * Team preparation: Day before. 1 hour.
241 * Trial 1:
242 ** Setup scenario: Test day 1. 09:00 - 09:30
243 ** Setup tech and perform checks: Test day 1. 09:00 - 09:30
244 ** Briefing of each team individually (outside building): Test day 1: 09:00 - 09:30
245 ** Mission execution: Test day 1. 09:30 - 10:30
246 ** Hot wash: Test day 1. 10:30-11:00
247 ** Scenario reset: test day 1. 10:30-11:00
248 * Teams switch location: Test day 1. 11:00-11:15
249 * Trial 2: 
250 ** (Setup scenario: Test day 1. 12:30 - 13:00)
251 ** Setup tech and perform checks: Test day 1. 12:30 - 13:00
252 ** Briefing of each team individually (outside building): Test day 1: 12:30 - 13:00
253 ** Mission execution: Test day 1. 13:00 - 14:00
254 ** Hot wash: Test day 1. 14:00-14:30
255 * Cleanup area and store tech. 14:00-14:30
256
257
258
259 **12. Materials**
260
261 * For FRs:
262 ** 1x Anymal with teleop setup and networking
263 ** 2x Tablet with C3I for squad leader (one spare)
264 ** 1x Base of Operations with computer with C3I
265 ** 4x Health sensors
266 ** 4x Location sensors
267 ** 8x walky talky with two channels (?) (for FR team and BoO).
268 * For usecase:
269 ** 2x training building (~~100m2?)
270 ** Enviromental challenges: smoke machine, obstacles, etc.
271 ** ~~8x Actors or dummies for victims.
272 * 8x Clipboards with papers and pens for hotwash and questionnaires
273
274 **13. Dependencies**
275
276 * Availability of materials
277 * Availability and functioning of ANYmal robot with teleoperation and network in target buildings
278 * Availability of functioning network between robot, FRs, and BoO.
279 * Availability of:
280 ** Experiment participants
281 ** People willing to play victim
282 ** Experiment leaders
283
284 **14. Success Criteria**
285 When mission(s) completed by both teams, and we can compare their performance using the metrics.
286
287
288