Changes for page g. Test CFT4 and the coming IFT's

Summary

• Page properties (1 modified, 0 added, 0 removed)

Details

Page properties

Content

...	...	@@ -1,64 +1,333 @@
1	1	= 1. Introduction =
2	2
3		-//<include a short summary of the claims to be tested, i.e., the effects of the functions in a specfic use case>//
	3	+This experiment validates multiple human-machine teaming technologies in Urban Search and Rescue (USAR) operations. Four operational modules simulate a full operational storyline across two days: wide area assessment, full-area reconnaissance with health monitoring, indoor drone-assisted search, and precision inspection in confined spaces.
4	4
5		-Claims uit [[UC01.1: Health and environmental monitoring (Firefighters)>>doc:2\. Specification.b\. Use Cases.UC01\.0\: Health Sensors.Usecase\: Health sensors (Firefighters).WebHome]] die getest worden:
	5	+The modules test system functions from five use cases, and aim to quantify effects on safety, situation awareness (SA), physical workload, mission effectiveness, and decision-making quality. Results will be compared against expected performance without these technologies, based on either baseline team data, observer input, or known solution benchmarks.
6	6
7		-* Weten ze waar tocix gas is?
8		-* commander weet wat de situatie van zijn personeel is?
9		-* Reddingsmedewerk heeft SA over hun eigen status (genoeg dat ze optijd kunnen reageren) (d.m.v. communicatie met commander of d.m.v. trillen sensor)?
10		-* HQ krijgt voldoende (en op het juiste moment) informatie over de situatie in het veld om ondersteuning te kunnen bieden?
	7	+----
11	11
12		-Claims uit UC01.2:Health and environmental monitoring (USAR) die getest worden:
	9	+= 2. Method =
13	13
14		-* Zelfde als hierboven maar dan iets aangepast voor USAR
	11	+== 2.1 Participants ==
15	15
	13	+Approximately 24–30 international first responders, organized in teams. Each team rotates across the four modules. Roles include responders, team leaders, drone/robot operators, analysts, medics, and safety officers.
16	16
	15	+== 2.2 Experimental Design ==
17	17
18		-Claims uit UC02.2: Indoor Drone Exploration and Victim Detection (USAR) die getest worden:
	17	+A **within-subject design** is used where all teams go through the four modules. Performance is compared across modules and against predefined baseline criteria. Observers collect data in real time; surveys and biometric data are used to validate subjective and objective measurements.
19	19
20		-* Weten first responders (genoeg) wat er binnen is om veilig naar binnen te gaan? hebben ze verhoogde SA van de binnenkant van een gebouw? SA/reliance
21		-* Kunnen er beter en sneller victims worden gevonden? > speed, task performance
22		-* Kunnen er meer betrouwbare analyses worden gemaakt van de binnenkant van een gebouw door bijv. een plan maken voor een veilige/ begaanbare route > SA
23		-* Task performance: kunnen er sneller en meer gestroomlijnd victim reports worden gemaakt en gedeeld (essentie = gaat victim assessement beter)?
	19	+----
24	24
	21	+== 2.3 Tasks (Per Module) ==
25	25
	23	+----
26	26
27		-Claims uit UC02.1: Indoor Drone Exploration and Victim Detection (Firefighters) die getest worden:
	25	+=== **Module 1 – Wide Area Assessment** ===
28	28
29		-* zelfde als hierboven maar dan iets meer aangepast voor Firefighters
	27	+**Use Case**: UC03.0
	28	+**Scenario**: Teams arrive at a simulated disaster zone. Structures are unstable. Drone support is requested for external mapping and hazard detection.
30	30
	30	+**Tested Functions**:
31	31
	32	+* Drone feed provides real-time visuals to field teams and command
	33	+* Zoom-ins allow inspection of rooftops and entry points
	34	+* Footage used to mark safe approach routes
32	32
	36	+**Measured Claims**:
33	33
34		-= 2. Method =
	38	+* CL1: Improved external SA
	39	+* CL2: Safer movement planning
	40	+* CL3: Faster planning cycle
	41	+* CL4: Reduced mental workload for recon
	42	+* CL5: Improved coordination (shared SA)
35	35
	44	+**Quantifiable Success Factors**:
36	36
37		-== 2.1 Participants ==
	46	+* ≥80% of hazards correctly marked on the map (based on preset dummy hazards)
	47	+* ≥90% agreement in SA between team and command (map match)
	48	+* Average planning time ≤ 10 minutes from drone launch
	49	+* NASA-TLX workload score ≤ 50 (moderate) for command roles
38	38
	51	+**How to Measure**:
39	39
40		-== 2.2 Experimental design ==
	53	+* Observer logs & stopwatch for planning time
	54	+* Map test: Compare team-drawn vs. actual map (SAGAT-lite)
	55	+* Count number of correctly identified hazards from drone feed
	56	+* NASA-TLX filled by drone operator and team lead
	57	+* Post-module survey: "How useful was the drone in forming your plan?" (1–5)
41	41
	59	+----
42	42
43		-== 2.3 Tasks ==
	61	+=== **Module 2 – Health Monitoring & Reconnaissance** ===
44	44
	63	+**Use Cases**: UC01.1 (Fire) and UC01.2 (USAR)
	64	+**Scenario**: Team performs full-area recon. Wearables measure heart rate, hydration, and simulated gas exposure. Simulated fatigue and alerts escalate to medics or team leads.
45	45
	66	+**Tested Functions**:
	67	+
	68	+* Alerts for fatigue/gas exposure
	69	+* Remote dashboard monitoring by safety officer
	70	+* Escalation protocols for health interventions
	71	+* Logging and after-action review
	72	+
	73	+**Measured Claims**:
	74	+
	75	+* CL1–CL2: Prevent overexertion and increase responder awareness
	76	+* CL3–CL4: Enable remote intervention and informed medical decision
	77	+* CL5: Enable better rotation/rest planning
	78	+* CL6: Debrief uses health logs
	79	+* CL7: Improve mission success
	80	+
	81	+**Quantifiable Success Factors**:
	82	+
	83	+* ≥90% of health alerts acknowledged within 1 minute
	84	+* ≥80% of interventions judged "timely" in AAR interviews
	85	+* ≥50% of teams adjust tactics or rest cycles based on health data
	86	+* ≥1 health-based lesson identified per team in debrief
	87	+* ≤2 simulated incidents due to unmanaged fatigue/gas exposure
	88	+
	89	+**How to Measure**:
	90	+
	91	+* Log alert timings vs. response time
	92	+* Observer notes + medic reports on intervention
	93	+* Exit survey: "Did alerts help prevent fatigue/injury?"
	94	+* Use of wearable dashboard during debrief (Yes/No)
	95	+* NASA-TLX for responders
	96	+
	97	+----
	98	+
	99	+=== **Module 3 – Indoor Drone Search (Barracks)** ===
	100	+
	101	+**Use Cases**: UC02.1 and UC02.2
	102	+**Scenario**: Collapsed barracks building. Indoor drone used for autonomous scan. Analyst tags victims, hazards, and updates C3I map. Drone does close inspection on request.
	103	+
	104	+**Tested Functions**:
	105	+
	106	+* Pre-entry thermal scan
	107	+* Hazard/victim detection
	108	+* Analyst-supported interpretation and tagging
	109	+* Entry planning based on drone data
	110	+
	111	+**Measured Claims**:
	112	+
	113	+* CL1: Heightened SA before entry
	114	+* CL2: Increased safety (less exposure)
	115	+* CL3–CL5: Faster, more accurate victim detection
	116	+* CL6: Trust in drone data
	117	+* CL7: Increased mission efficiency
	118	+
	119	+**Quantifiable Success Factors**:
	120	+
	121	+* ≥90% of dummy victims detected by drone+analyst
	122	+* ≥2 new hazards marked per team from drone feed
	123	+* Average time-to-first victim ≤ 3 minutes
	124	+* ≥80% of responders rate drone info as “trustworthy” (score ≥4/5)
	125	+* ≤1 injury due to unknown hazard in follow-up entry
	126	+
	127	+**How to Measure**:
	128	+
	129	+* Victim tags placed in known positions for ground truth
	130	+* Observer logs: detection times and analyst confirmations
	131	+* Team SA quiz: "How many victims? Where were they located?"
	132	+* Trust survey: “I would act on this drone data” (1–5)
	133	+* Entry path compared to drone hazard map
	134	+
	135	+----
	136	+
	137	+=== **Module 4 – Precision Inspection with ANYMAL/SNAKE** ===
	138	+
	139	+**Use Case**: UC04.0
	140	+**Scenario**: Teams reach unstable voids. Robots are deployed to inspect inaccessible areas. SNAKE arm is used to look into cracks. Results update team maps and entry plans.
	141	+
	142	+**Tested Functions**:
	143	+
	144	+* Autonomous or manual ANYMAL movement
	145	+* Void inspection using flexible arm
	146	+* Victim/hazard confirmation
	147	+* Decision-making based on robot visuals
	148	+
	149	+**Measured Claims**:
	150	+
	151	+* CL1: Access without risk
	152	+* CL2: Detection in confined space
	153	+* CL3: Safer routing
	154	+* CL4: Trust in robot-assessed visuals
	155	+* CL5: Faster room clearing
	156	+
	157	+**Quantifiable Success Factors**:
	158	+
	159	+* ≥2 hazards or victims confirmed via SNAKE per team
	160	+* ≥80% of voids scanned without human entry
	161	+* ≥70% of teams adjust route based on robot findings
	162	+* ≥80% of participants rate robot visuals as “clear and usable”
	163	+* Average inspection time ≤ 8 minutes per room
	164	+
	165	+**How to Measure**:
	166	+
	167	+* Observer log: robot path vs. human path
	168	+* Detection log compared to known hidden items
	169	+* Survey: “Did robot findings improve your plan?” (Yes/No)
	170	+* Video review of time-per-room
	171	+* Trust in visuals scale (1–5)
	172	+
	173	+
	174	+
46	46	== 2.4 Measures ==
47	47
	177	+This section describes how each claim will be measured during each module, using a combination of objective logging, observer annotations, post-task surveys, and scenario-based evaluation.
48	48
	179	+----
	180	+
	181	+=== **Module 1 – Wide Area Assessment (UC03.0)** ===
	182	+
	183	+|=**Claim**|=**Metric**|=**Method/Tool**|=**Success Threshold**
	184	+|CL1 – Improved SA|Number of hazards correctly identified on team maps|SAGAT-lite: Pre/post map-drawing task + verbal hazard recall|≥80% match with ground-truth hazard list
	185	+|CL2 – Safer planning|Number of hazard zones avoided during later entry|Observer logs cross-referenced with hazard map|100% of marked hazards avoided
	186	+|CL3 – Faster planning|Time from drone launch to team briefing|Stopwatch & observer notes|≤10 minutes total
	187	+|CL4 – Reduced workload|Mental workload score of command & drone operator|NASA-TLX (short form)|≤50 average score
	188	+|CL5 – Shared SA|Consistency between team and command in map data|Comparison of annotations across roles|≥90% agreement on key features
	189	+
	190	+
	191	+
	192	+----
	193	+
	194	+=== **Module 2 – Health Monitoring & Reconnaissance (UC01.1 / UC01.2)** ===
	195	+
	196	+|=**Claim**|=**Metric**|=**Method/Tool**|=**Success Threshold**
	197	+|CL1 – Prevent overload|HR trend + alert timing vs. pause/extraction|Wearable logs + observer notes|≥90% alerts followed by correct action within 1 minute
	198	+|CL2 – Responder awareness|Survey response on self-adjustment|Post-task Likert: “The alert helped me act”|≥80% rate 4 or 5
	199	+|CL3 – Remote escalation|Alert-to-medic contact time|System log + stopwatch|≤1 minute average
	200	+|CL4 – Medical support|Alignment of alerts with medical assessment|Medic forms + sensor log correlation|≥80% concordance
	201	+|CL5 – Operational planning|Number of rest/rotation decisions based on dashboard|Observer logs + team lead AAR|≥50% of teams adapt plan
	202	+|CL6 – AAR use of health data|Was biometric data used during debrief?|Debrief analysis|Yes, per team
	203	+|CL7 – Mission effectiveness|Task time + incidents avoided|Stopwatch + incident log|Task time not slower than baseline; 0 uncontrolled fatigue/gas incidents
	204	+
	205	+
	206	+
	207	+----
	208	+
	209	+=== **Module 3 – Indoor Drone Search (UC02.1 / UC02.2)** ===
	210	+
	211	+|=**Claim**|=**Metric**|=**Method/Tool**|=**Success Threshold**
	212	+|CL1 – Heightened SA|SA questionnaire + map task|Pre/post: victims, layout, hazard count|≥80% correct recall post-drone
	213	+|CL2 – Increased safety|Hazard zone avoidance rate|Observer vs. ground truth map|≥90% of flagged areas avoided
	214	+|CL3 – Faster victim detection|Time to first detection|Stopwatch from drone entry|≤3 minutes
	215	+|CL4 – Accuracy of detection|Victim detection rate|Drone log vs. planted victims|≥90% detected
	216	+|CL5 – Trust in results|Survey: “I trust the drone data for decision-making”|1–5 Likert scale|≥80% rate 4 or 5
	217	+|CL6 – Efficiency|Entry time after drone plan vs. without drone|Stopwatch; compare with baseline data|10–20% faster planning phase
	218	+
	219	+
	220	+
	221	+----
	222	+
	223	+=== **Module 4 – Robot-Based Precision Inspection (UC04.0)** ===
	224	+
	225	+|=**Claim**|=**Metric**|=**Method/Tool**|=**Success Threshold**
	226	+|CL1 – Extended reach|Percentage of voids explored by robot not human|Observer log + inspection plan|≥80% of voids scanned by robot
	227	+|CL2 – Detection in small spaces|Victim/hazard detection in hidden locations|Camera log vs. planted markers|≥2 findings per team
	228	+|CL3 – Safer routing|Route changes based on robot input|Pre/post plan comparison + observer notes|≥70% of teams adapt plan
	229	+|CL4 – Trust in visuals|Survey on clarity and trust in robot data|Likert: “The robot data was sufficient for decisions”|≥80% rate 4 or 5
	230	+|CL5 – Room clearing speed|Time per room before vs. after robot scout|Stopwatch log|≤8 minutes per room avg.
	231	+
	232	+
	233	+
	234	+----
	235	+
49	49	== 2.5 Procedure ==
50	50
51		-Ttijdens debriefing vragen of ze bepaalde dingen hebben gemerkt
	238	+All modules follow a similar four-part procedure, tailored per use case.
52	52
	240	+=== **General Daily Timeline** ===
	241	+
	242	+* **08:30 – 09:00**: Morning briefing, safety, tech setup
	243	+* **09:00 – 12:00**: First module rotation (two parallel teams)
	244	+* **13:00 – 16:00**: Second module rotation (two parallel teams)
	245	+* **16:00 – 17:00**: Shared after-action review
	246	+
	247	+Each module runs with the following structure:
	248	+
	249	+=== **Per Module Procedure** ===
	250	+
	251	+1. (((
	252	+**Briefing (10–15 min)**
	253	+
	254	+* Explain objectives, scenario, roles, safety, success factors
	255	+* Introduce technology and expectations
	256	+)))
	257	+1. (((
	258	+**Execution Phase (45–60 min)**
	259	+
	260	+* Scenario runs in real time
	261	+* Observer logs events, actions, communications
	262	+* System logs recorded (drone, robot, wearables)
	263	+)))
	264	+1. (((
	265	+**Measurement Phase (15–20 min)**
	266	+
	267	+* Paper or tablet surveys: SA, trust, NASA-TLX
	268	+* Sensor data downloaded to central system
	269	+* Short interview or checklist with operator and team lead
	270	+)))
	271	+1. (((
	272	+**Debrief (15–20 min)**
	273	+
	274	+* Team reflects on use of technology, decision-making
	275	+* Facilitator prompts discussion of claims (trust, effectiveness, awareness)
	276	+* Recorded notes for final reporting
	277	+)))
	278	+
	279	+For cross-checking performance without the tech, one team per module may be assigned a simplified "control" version of the scenario, using conventional tools only (where feasible).
	280	+
	281	+----
	282	+
53	53	== 2.6 Material ==
54	54
	285	+Each module requires scenario-specific equipment, environmental props, logging tools, and survey forms:
55	55
56		-= 3. Results =
	287	+=== **Common Materials (all modules)** ===
57	57
	289	+* Observer logbooks (standardized per module)
	290	+* Stopwatch or time-tracking app
	291	+* Participant role badges and checklists
	292	+* Data collection station with tablets/laptops
	293	+* Printed Likert-scale surveys (SA, trust, workload)
	294	+* SAGAT-lite map templates
58	58
59		-= 4. Discussion =
	296	+=== **Module-Specific Materials** ===
60	60
	298	+**Module 1 – Wide Area Assessment**
61	61
62		-= 5. Conclusions =
	300	+* Outdoor drones with RTK GPS and live zoom cameras
	301	+* Command screen with drone feed
	302	+* Large printed site maps with hazard zones (for scoring)
	303	+* Structural hazard props (collapsed façades, signs)
63	63
64		-
	305	+**Module 2 – Health Monitoring**
	306	+
	307	+* Wearable sensors (HR, hydration, gas; real or simulated)
	308	+* Dashboard software for live feed + logging
	309	+* Incident trigger devices (e.g., CO2 canisters, alarms)
	310	+* Medic checklist sheets
	311	+* Alert simulation software (optional)
	312	+
	313	+**Module 3 – Indoor Drone Search**
	314	+
	315	+* Thermal indoor drone with autonomous mode
	316	+* C3I-compatible map annotation system
	317	+* Dummy victims with heat packs or QR markers
	318	+* Printed room layouts for SA testing
	319	+* Indoor hazard props (rubble, fake smoke, blocked doors)
	320	+
	321	+**Module 4 – ANYMAL and SNAKE**
	322	+
	323	+* ANYMAL robot (legged) and SNAKE articulated arm
	324	+* Confined space mockups (voids, crawlspaces, stairs)
	325	+* Hidden hazard/victim tags inside small cavities
	326	+* Robot operator station + external monitor
	327	+* Scenario map with route overlays
	328	+
	329	+= 3. Results =
	330	+
	331	+= 4. Discussion =
	332	+
	333	+= 5. Conclusions =