Changes for page k. Design Patterns Evaluation SFT1
Last modified by Tjalling Haije on 2026/07/27 16:18
From version 3.1
edited by Tjalling Haije
on 2026/07/27 13:37
on 2026/07/27 13:37
Change comment:
There is no comment for this version
To version 4.4
edited by Tjalling Haije
on 2026/07/27 15:03
on 2026/07/27 15:03
Change comment:
There is no comment for this version
Summary
-
Page properties (1 modified, 0 added, 0 removed)
-
Attachments (0 modified, 3 added, 0 removed)
Details
- Page properties
-
- Content
-
... ... @@ -65,7 +65,13 @@ 65 65 === 2. Incident 2 tech evaluation === 66 66 67 67 |(% style="width:247px" %)**Tech**|(% style="width:319px" %)**How was it used**|(% style="width:329px" %)**Issues**|(% style="width:329px" %)**Did it work sufficiently?** 68 -|(% style="width:247px" %)[[ASTRAL RTK Drone>>Main.c\. Technology.ASTRAL RTK Drone.WebHome]] (outdoor)|(% style="width:319px" %)Baseline & Run 1-3: The drone team conducts an aerial assessment of the collapsed buildings, identifying structural risks, access points, and possible victim locations. |(% style="width:329px" %)Run 1 USAR 1: Drone had connection issues which delayed its deployment.|(% style="width:329px" %)Yes 68 +|(% style="width:247px" %)[[ASTRAL RTK Drone>>Main.c\. Technology.ASTRAL RTK Drone.WebHome]] (outdoor)|(% style="width:319px" %)Baseline & Run 1-3: The drone team conducts an aerial assessment of the collapsed buildings, identifying structural risks, access points, and possible victim locations. |(% style="width:329px" %)((( 69 +Run 1 USAR 1: Drone had connection issues which delayed its deployment. 70 + 71 +{{warning}} 72 +TODO: expand these issues with questionnaire answers 73 +{{/warning}} 74 +)))|(% style="width:329px" %)Yes 69 69 |(% style="width:247px" %)[[OWL Indoor Drone>>Main.c\. Technology.OWL Indoor Drone.WebHome]] |(% style="width:319px" %)((( 70 70 Run 2 USAR 2: used to explore building and search for objects and victims 71 71 ... ... @@ -73,7 +73,9 @@ 73 73 )))|(% style="width:329px" %)((( 74 74 Run 2 USAR 1: OWL missed victim and entire floor. Object detection initially disabled. 75 75 76 - 82 +{{warning}} 83 +TODO: expand these issues with questionnaire answers 84 +{{/warning}} 77 77 )))|(% style="width:329px" %)Unreliable 78 78 |(% style="width:247px" %)[[ANYmal Robot>>Main.c\. Technology.Humanoid Robot.WebHome]] with [[Robot Arm (for ANYmal)>>Main.c\. Technology.Robot Arm (for ANYmal).WebHome]] |(% style="width:319px" %)((( 79 79 Run 1 USAR 1: remove beam blocking door ... ... @@ -81,8 +81,10 @@ 81 81 Run 3 USAR 1: Used to close a valve 82 82 83 83 84 -)))|(% style="width:329px" %) |(% style="width:329px" %)Yes 85 -|(% style="width:247px" %)[[ANYmal Robot>>Main.c\. Technology.Humanoid Robot.WebHome]] with [[SNAKE>>Main.c\. Technology.SNAKE.WebHome]] |(% style="width:319px" %)Run 3 USAR 1: Used to search for victims in a narrow space|(% style="width:329px" %) |(% style="width:329px" %)? 92 +)))|(% style="width:329px" %){{warning}}TODO: expand these issues with questionnaire answers{{/warning}}((( 93 + 94 +)))|(% style="width:329px" %)Yes 95 +|(% style="width:247px" %)[[ANYmal Robot>>Main.c\. Technology.Humanoid Robot.WebHome]] with [[SNAKE>>Main.c\. Technology.SNAKE.WebHome]] |(% style="width:319px" %)Run 3 USAR 1: Used to search for victims in a narrow space|(% style="width:329px" %){{warning}}TODO: expand these issues with questionnaire answers{{/warning}}|(% style="width:329px" %)? 86 86 87 87 === 3. Incident 2 way of working === 88 88 ... ... @@ -101,17 +101,34 @@ 101 101 102 102 == 4. Raw results == 103 103 104 - **Results for Incident 2, Run1, USAR team 1**114 +=== Results for Incident 2, Run 3, USAR team 1 VS baseline USAR team 1 === 105 105 116 +Analysis of questionnaire results for members with ID: UT13, UT11, UT12, UTL1 117 + 118 + 119 +|=Scenario|=Role|=Participants 120 +|Baseline|USAR Team Member|3 121 +|Baseline|USAR Team Leader|1 122 +|Run 3|USAR Team Member|3 123 +|Run 3|USAR Team Leader|1 124 + 125 +[[image:1785164456543-963.png||height="376" width="752"]] 126 + 127 +[[image:1785164494950-855.png||height="348" width="756"]] 128 + 129 +[[image:1785164507435-933.png||height="253" width="745"]] 130 + 131 + 106 106 Note: The measures are defined in [[UC04.3: Detailed indoor exploration with ANYMAL (USAR)>>2\. Specification.b\. Use Cases.UC04\.0\: Detailed indoor exploration.UC04\.3\: Detailed indoor exploration with ANYMAL (USAR).WebHome]] and [[x. Measures>>3\. Evaluation.Measures.WebHome]] 107 107 108 108 |(% style="width:376px" %)**Measure**|(% style="width:225px" %)**Results**|**Connected Claim**|**Claim proven?** 109 109 |(% style="width:376px" %)Safe path chosen|(% style="width:225px" %) |CL01 Improve safety for responders | 110 110 |(% style="width:376px" %)Decision quality|(% style="width:225px" %) |CL01 Improve safety for responders | 111 -|(% style="width:376px" %)Near-incident and incident reports|(% style="width:225px" %) |CL01 Improve safety for responders | 112 -|(% style="width:376px" %)Reported hazards, victims, layout correctness, based on robot intel before FR entry|(% style="width:225px" %) |CL02 Increased SA| 113 -|(% style="width:376px" %)How optimal was the FR path|(% style="width:225px" %) |CL02 Increased SA| 114 -|(% style="width:376px" %)Reported hazards, victims, layout correctness, after FR entry|(% style="width:225px" %) |CL02 Increased SA| 137 +|(% style="width:376px" %)Near-incident and incident reports|(% style="width:225px" %)No incidents in baseline or run. 138 +No near-incidents data available.|CL01 Improve safety for responders |Unknown 139 +|(% style="width:376px" %)Reported hazards, victims, layout correctness, based on robot intel before FR entry|(% style="width:225px" %)No ground truth data available.|CL02 Increased SA|Unknown 140 +|(% style="width:376px" %)How optimal was the FR path|(% style="width:225px" %)No FR path or ground truth data available.|CL02 Increased SA|Unknown 141 +|(% style="width:376px" %)Reported hazards, victims, layout correctness, after FR entry|(% style="width:225px" %)No FR path or ground truth data available.|CL02 Increased SA|Unknown 115 115 |(% style="width:376px" %)Workload: e.g. NASA-TLX|(% style="width:225px" %) |CL03 improved mission effectiveness| 116 116 |(% style="width:376px" %)Appropriate trust in technology: e.g. trust survey|(% style="width:225px" %) |CL04 acceptable workload| 117 117 |(% style="width:376px" %)Decision confidence: e.g. Retrospective decision quality rating|(% style="width:225px" %) |CL04 acceptable workload|
- 1785164456543-963.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.TjallingHaije - Size
-
... ... @@ -1,0 +1,1 @@ 1 +146.4 KB - Content
- 1785164494950-855.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.TjallingHaije - Size
-
... ... @@ -1,0 +1,1 @@ 1 +162.9 KB - Content
- 1785164507435-933.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.TjallingHaije - Size
-
... ... @@ -1,0 +1,1 @@ 1 +108.5 KB - Content