Skip to Content
Last modified by Rosa Van Tuijn on 2026/04/13 12:47
From version 2.2
edited by Rosa Van Tuijn
on 2026/03/24 16:00
Change comment: There is no comment for this version
To version 1.14
edited by Tjalling Haije
on 2026/03/23 15:22
Change comment: There is no comment for this version

Summary

Details

Page properties
Title
... ... @@ -1,1 +1,1 @@
1 -h. Test SFT1: System baseline test
1 +h. Test SFT1
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.RosaVanTuijn
1 +XWiki.TjallingHaije
Content
... ... @@ -1,5 +1,5 @@
1 1  **Experiment title**
2 -System baseline test
2 +[Short, descriptive title]
3 3  
4 4  
5 5  **2. Objective / Research Question**
... ... @@ -10,6 +10,7 @@
10 10  1. FR Health
11 11  1. FR and victim safety
12 12  
13 +
13 13  **3. Hypotheses / Expectations (optional)**
14 14  See the claims from [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]], being:
15 15  
... ... @@ -21,6 +21,7 @@
21 21  * CL06: improved FR health
22 22  * CL07: degraded mission efficiency
23 23  
25 +
24 24  **4. Scenario / Context**
25 25  ASR3: Detailed indoor exploration
26 26  
... ... @@ -32,6 +32,7 @@
32 32  ** **ANYmal **is mission-ready (battery, controls, and payload checked).
33 33  ** **C3I is online**, role-based views are configurated (Operator/ Analyst/ Team leader/ Safety officer); **5G pods** are installed for connection.
34 34  
37 +
35 35  **5. Participants**
36 36  
37 37  * FR team ROBOT:
... ... @@ -93,6 +93,7 @@
93 93  
94 94  * Building A: Easier location, obstacles and environmental conditions. (clear visibility, ..)
95 95  * Building B: Challenging location, obstacles and environmental conditions for humans and robots (e.g. smoke and obstacles).
99 +*
96 96  
97 97  Trial 1:
98 98  
... ... @@ -104,61 +104,25 @@
104 104  * FR Team ROBOT does Building B
105 105  * FR team HUMAN does Building A
106 106  
111 +
107 107  **9. Measurements**
108 108  
109 -See the measurements from [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]]. Compare outcomes between the two conditions teams if applicable.
114 +See the measurements from [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]]
110 110  
111 -|**Claim: Safety**|**Measurment(s)**|**How to test **
112 -|Safer decision before entry|Safe path chosen|(((
113 -* Define ground-truth safe path(s) beforehand so that they can be compared to chosen entry path during the trial
114 -* Score paths beforehand: safe/ suboptimal/ unsafe
115 -)))
116 -|Better decision overall|"How good were you decisions looking back?"|(((
117 -* Hotwash reflection question (likert + open)
118 -* Observations of errors (e.g. near-incidents, unsafe choices)
119 -)))
120 -|Fewer dangerous situations|Near-incident reports|(((
121 -* Define hazards in buildings beforehand and check (with observations of during hotwash) if these hazards are avoided or not
122 -)))
123 123  
124 -|**Claim: Efficiency**|**Measurment(s)**|**How to test **
125 -|Faster mission execution|Total mission completion time|(((
126 -* Timestamp of start (after briefing) till all located victims have been extracted
127 -)))
128 -|Faster inside performance|First responder time inside|(((
129 -* Timestamp of entry building till exist building (last located victim is exstracted)
130 -)))
131 -
132 -|**Claim: Situation Awareness**|**Measurment(s)**|**How to test **
133 -|Before entry |(((
134 -Correctness of reported hazards, victims, layout
135 -)))|(((
136 -
137 -)))
138 -|During entry|(((
139 -FR path logs
140 -)))|(((
141 -
142 -)))
143 -|After mission|(((
144 -Correctness of reported hazards, victims, layout
145 -)))|
146 -
147 -
148 -
149 149  **10. Procedure (Step-by-Step)**
150 150  
151 -1. __Team preparation__:
119 +1. Team preparation:
152 152  11. FR team ROBOT is trained in operating the robot, analysing the camera stream, and pinning information on C3I
153 153  11. Squad leader and BoO are trained for both teams in interpreting the health and location sensor data.
154 154  11. Experiment leader knows where hazards and victims are in both buildings.
155 155  11. Anymal robot is tested with network in target buildings for experiment (without FR themselves seeing building!)
156 -1. __Experiment preparation:__
124 +1. Experiment preparation
157 157  11. Building is prepped with victims, hazards, etc.
158 -1. __Briefing:__
126 +1. Briefing:
159 159  11. Same for both teams for both building types: Building has been inspected from outside and is estimated safe for entry. Situation inside unknown. Victims possibly present. Goal: find, assess, and extract victims while avoiding and mapping hazards.
160 160  11. One person is notified that their health sensor will be triggered during the Building B (difficult building) situation.
161 -1. __Execution:__
129 +1. Execution:
162 162  11. For FR team ROBOT, see: [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]]
163 163  11. For FR team HUMAN, see: [[UC04.4: Detailed indoor exploration without ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYmal (USAR).WebHome]]
164 164  11. During execution of both teams, performance will be measured by experiment leader:
... ... @@ -166,38 +166,37 @@
166 166  111. Health indicators stay within acceptable range
167 167  111. Time for tackling any health alerts
168 168  111. Track FR path
169 -1. __Hotwash__:
170 -11. Questions (specified per role, couple questions per topic) on:
137 +1. Hotwash:
138 +11. Questions on:
171 171  111. Near-incident report
172 172  111. Path inside
173 173  111. FR location and status of hazards, victims, and layout
174 -111. Mission effectiveness: e.g. self reported effectiveness of decisions
142 +111. Mission effectiveness
175 175  111. Self-reported workload
176 176  111. Trust survey
177 -11. Group discussion with feedback
178 178  
179 -**~11. Planning:**
146 +Planning:
180 180  
181 181  * Team preparation: Day before. 1 hour.
182 182  * Trial 1:
183 -** Setup scenario: Test day 1. 09:00 - 09:30
150 +** Setup usecase: Test day 1. 09:00 - 09:30
184 184  ** Setup tech and perform checks: Test day 1. 09:00 - 09:30
185 185  ** Briefing of each team individually (outside building): Test day 1: 09:00 - 09:30
186 186  ** Mission execution: Test day 1. 09:30 - 10:30
187 187  ** Hot wash: Test day 1. 10:30-11:00
188 -** Scenario reset: test day 1. 10:30-11:00
155 +** Usecase reset: test day 1. 10:30-11:00
189 189  * Teams switch location: Test day 1. 11:00-11:15
190 190  * Trial 2: 
191 -** (Setup scenario: Test day 1. 12:30 - 13:00)
158 +** Setup usecase: Test day 1. 12:30 - 13:00
192 192  ** Setup tech and perform checks: Test day 1. 12:30 - 13:00
193 193  ** Briefing of each team individually (outside building): Test day 1: 12:30 - 13:00
194 194  ** Mission execution: Test day 1. 13:00 - 14:00
195 195  ** Hot wash: Test day 1. 14:00-14:30
196 -* Cleanup area and store tech. 14:00-14:30
163 +** Cleanup area and store tech. 14:00-14:30
197 197  
198 198  
199 199  
200 -**12. Materials**
167 +**~11. Materials**
201 201  
202 202  * For FRs:
203 203  ** 1x Anymal with teleop setup and networking
... ... @@ -212,10 +212,11 @@
212 212  ** ~~8x Actors or dummies for victims.
213 213  * 8x Clipboards with papers and pens for hotwash and questionnaires
214 214  
215 -**13. Dependencies**
216 216  
183 +**14. Dependencies**
184 +
217 217  * Availability of materials
218 -* Availability and functioning of ANYmal robot with teleoperation and network in target buildings
186 +* Availaibility and functioning of ANYmal robot with teleoperation and network in target buildings
219 219  * Availability of functioning network between robot, FRs, and BoO.
220 220  * Availability of:
221 221  ** Experiment participants
... ... @@ -222,8 +222,9 @@
222 222  ** People willing to play victim
223 223  ** Experiment leaders
224 224  
225 -**14. Success Criteria**
226 -When mission(s) completed by both teams, and we can compare their performance using the metrics.
227 227  
194 +**15. Success Criteria**
195 +When mission completed by both teams, and we can compare their performance.
228 228  
197 +
229 229