chr_schwarz
|
22
|
1
|
Cubic-notes from usability test
|
|
2
|
-------------------------------
|
chr_schwarz
|
31
|
3
|
For 1.0 tutorial:
|
chr_schwarz
|
22
|
4
|
Does Common apply to more than one test?
|
|
5
|
ExPoint hard to understand ==> Tutorial
|
|
6
|
None used subtest! ==> Tutorial
|
|
7
|
Confusing ExPoint vs Tree ==> Tutorial
|
|
8
|
Some use exStartPoint instad of tree test. ==> Explain what to use when
|
|
9
|
Can more than one test start from ExPoint?
|
|
10
|
Tree test should be far more visible.
|
|
11
|
Tree test was hard to understand
|
|
12
|
Wants to remember user input / info for next steps. E.g. which fund is selected here.
|
|
13
|
It was not understood that "Text" could be used for many different things.
|
|
14
|
How to test error scenarios / input validation? ==> Tutorial/guide?
|
|
15
|
Confused which pages was the same when using ExStartPoint
|
|
16
|
ExStartPoint, they are often duplicating content from originating page (want content from source page)
|
|
17
|
Users want to google for cubic hints / usage guide..
|
|
18
|
IdType hard to understand
|
|
19
|
Training is necessary for effective use
|
|
20
|
F1 help should be available
|
|
21
|
Example site and tests are very important
|
|
22
|
Not evident that all elements can be used for user action
|
|
23
|
Wants dynamic number of page elements (e.g. checkboxes)
|
|
24
|
Info abount sequence (info that not important?) - it is relevant
|
skytteren
|
2
|
25
|
|
chr_schwarz
|
31
|
26
|
For 2.0 tutorial
|
chr_schwarz
|
22
|
27
|
What is i18n and params?
|
|
28
|
|
|
29
|
|
|
30
|
Observations/questions/opinions:
|
chr_schwarz
|
31
|
31
|
--------------------------------
|
chr_schwarz
|
22
|
32
|
M3 Most users right click often to see what is possible
|
|
33
|
M3 What is Extension point? (hard to understand. Can several tests extend from one point?)
|
|
34
|
M3 Messed with IdText etc in property view ==> tooltips?
|
|
35
|
M3 Wants to enter text into checkbox user action => Remove header?
|
|
36
|
M3 Tries to put extension points on canvas, not on test. => Anonymous, then gets name from page
|
chr_schwarz
|
31
|
37
|
(Cycles in test was *very* popular, flow diagram style)
|
|
38
|
(Own view for customers? (e.g. not property view and not contexts))
|
|
39
|
(ExStartPoint should be more flexible (be possible to create user interactions from it?))
|
chr_schwarz
|
22
|
40
|
|
|
41
|
|
chr_schwarz
|
31
|
42
|
General:
|
|
43
|
--------
|
chr_schwarz
|
35
|
44
|
Done The open file should be highlighted
|
|
45
|
Done Invoke "IDE.saveAllEditors" before create new test from extension point
|
chr_schwarz
|
22
|
46
|
M4 How to specify popup and alerts
|
|
47
|
M4? Should be able to test PDF (at least existance)
|
chr_schwarz
|
31
|
48
|
2.0 Should be some form of refactoring support (select states -> extract etc)
|
|
49
|
2.0 Provide a "TODO" element
|
chr_schwarz
|
35
|
50
|
(Confusing package explorer. Only interested in tests in the beginning. -> Better now that open file is highlighted).
|
chr_schwarz
|
31
|
51
|
(Users want to model cycles (go back to a state) ==> Show error message or allow? Use sequence numbers?)
|
chr_schwarz
|
22
|
52
|
(Test that print dialog box shows)
|
|
53
|
(Have a "shared repository" of page elements. => Use statistics)
|
|
54
|
|
|
55
|
|
chr_schwarz
|
31
|
56
|
Files and persistance:
|
|
57
|
----------------------
|
chr_schwarz
|
22
|
58
|
M3 Move .aat file, problems with SubTest and ExStartPoint (refactoring support) => Update path on move
|
|
59
|
M3 Move open .aat file, save again --> duplicated file ==> Close file before moving? (Eclipse hook)
|
|
60
|
2.0 Right click on file -> Run... (standard Eclipse menu) --> Run in Watir, etc.
|
|
61
|
|
chr_schwarz
|
31
|
62
|
Page:
|
|
63
|
-----
|
chr_schwarz
|
38
|
64
|
Done Header: Better contrast
|
chr_schwarz
|
97
|
65
|
Done Resize vertically on drag'n'drop (bug)
|
chr_schwarz
|
22
|
66
|
M3 Context menu for adding page elements
|
|
67
|
M3 Context meny for adding user interaction
|
chr_schwarz
|
97
|
68
|
(Wider figure, Auto resize width)
|
chr_schwarz
|
22
|
69
|
|
chr_schwarz
|
31
|
70
|
Contexts:
|
|
71
|
---------
|
chr_schwarz
|
94
|
72
|
Done What is Page Section? (No association to DIV, but tooltip was clearifying) => Change name
|
|
73
|
Done Wants to identify a single element, not an entire section.
|
chr_schwarz
|
97
|
74
|
M3 Problem with properties view of context with page element in it
|
chr_schwarz
|
94
|
75
|
M3? Provide coulumn headers
|
chr_schwarz
|
22
|
76
|
M4? Wants anonymous elements (amount, date etc, only that they are present).
|
|
77
|
M4? Have "List" as own context, not just table => Rename to Table/List??
|
|
78
|
(Watir and Selenium does not support contexts)
|
|
79
|
|
|
80
|
|
chr_schwarz
|
31
|
81
|
User interaction:
|
|
82
|
-----------------
|
chr_schwarz
|
94
|
83
|
Done Problem with drag and drop when holding down mouse button. Must click twice.
|
|
84
|
Done The "auto-select default action" feature (e.g. "Click") was a bit confusing => Use event
|
|
85
|
Done Checkbox icon on "uncheck"-action is wrong in the user interaction figure (bug)
|
chr_schwarz
|
97
|
86
|
M3 Use command for edit of properties
|
chr_schwarz
|
22
|
87
|
M3 Put --Choose-- automatically, not button "Add new"
|
|
88
|
M3 When two elements have the same description, display idText in user action dialog (tooltip)
|
|
89
|
M3 Put options in the "value" dropdown of select list.
|
|
90
|
M3 "Delete row"-button in input dialog
|
|
91
|
M3 When form element text changes, must update label on connection automatically => subscriptions must be examined
|
|
92
|
M3 "Move up" and "Move down" button wanted. Is sequence significant?
|
|
93
|
|
|
94
|
|
|
95
|
Palette:
|
chr_schwarz
|
31
|
96
|
--------
|
chr_schwarz
|
94
|
97
|
Done Add tooltips to all controls (e.g. RadioButton and Title)
|
chr_schwarz
|
97
|
98
|
Done Drag and drop: Too many clicks
|
|
99
|
Done "Add text" was confusing. ==> better "Add/check text"
|
|
100
|
Done Better structure (indenting?) of palette (e.g. Button was often not found)
|
|
101
|
Done Icons for all elements in palette
|
|
102
|
(Palette is too big for 1024 x 768)
|
|
103
|
(Be possible to turn off text in palette, only see icons + tooltip?)
|
chr_schwarz
|
22
|
104
|
|
|
105
|
|
|
106
|
Test editor:
|
chr_schwarz
|
31
|
107
|
------------
|
chr_schwarz
|
97
|
108
|
Done Not marquee as own control, better contain it in select (should work by default, but is not the case)
|
|
109
|
Done Problem with "sticky" Connection control. ==> Only sticky when source is Common
|
|
110
|
Done Problem with delete of page from URL start point
|
|
111
|
Done Issues with delete of SelectBox (?)
|
|
112
|
M3 Limit context menu of elements
|
|
113
|
M3 Use command for copy/cut/paste
|
chr_schwarz
|
22
|
114
|
M3 Mark elements with ctrl key
|
|
115
|
M3 Right-click on ExStartPoint -> "Open test" (none double clicked)
|
|
116
|
M3 "Cut" on Page does not work
|
|
117
|
M3 Copy, delete Page has some problems with connections
|
|
118
|
M3 Undo on "Cut" does not work => Use command stack
|
|
119
|
M3 ID on page elements was deleted when description was set (empty desc first?) (bug)
|
|
120
|
M3 Select all -> ctrl-x had problems / not implemented
|
|
121
|
M3 When holding ctrl key --> Move should become "Copy-paste"
|
|
122
|
M3 Copy Common, paste, then connection --> bug (connections stacked in background, wrong placement)
|
|
123
|
M3 Issues with delete of SelectBox (?)
|
chr_schwarz
|
31
|
124
|
M4? Bigger canvas, should be easier to extend (place elements at edge) (bug in GEF?)
|
|
125
|
(Draw line from link --> Auto UserAction on link)
|
|
126
|
(Easier "not present")
|
|
127
|
(Drag and drop element to another *test* (refactoring))
|
|
128
|
(Show page name from extenison point (?))
|
chr_schwarz
|
22
|
129
|
|
|
130
|
|
|
131
|
HTML export:
|
|
132
|
2.0? Have a ID on page, such that HTML exporter can merge different tests to better site?
|
|
133
|
|
|
134
|
|
|
135
|
|
|
136
|
=============================================
|
|
137
|
Previous TODOs:
|
|
138
|
=============================================
|
|
139
|
|
|
140
|
|
|
141
|
Christian suggests
|
|
142
|
-------------------
|
chr_schwarz
|
38
|
143
|
- Flag for marking that test is spec and not regression test (such that the build is not broken)
|
chr_schwarz
|
22
|
144
|
- Automatically create page when user interaction is about to be created and there are no target pages available.
|
|
145
|
Create new page with name from Link or form params(?)
|
|
146
|
- Make it possible to create extension start point from link on page with extension point.
|
|
147
|
- Label on userActions should be possible to place manually (or edit properties on placement).
|
|
148
|
- Make "Row" the default text for RowContext?
|
|
149
|
- BUG (fixed for M3): Not-link did not work
|
|
150
|
- Button for creating watir test (or context menu of test)
|
|
151
|
- Be able to assert cells from column headers
|
|
152
|
- With tree-test, use browser back instead of whole test again? Property?
|
|
153
|
- BUG: Options are not asserted correctly? Only user interaction on options correct?
|
|
154
|
- Be possible to start test from failed test ("Fix the test")
|
|
155
|
- Select box: Not center aligned?
|
|
156
|
- HTML skeleton: Checkbox bug? (when checked required)
|
|
157
|
- HTML skeleton: Alignment error (transition taking up too much horizontal space)
|
|
158
|
|
|
159
|
|
skytteren
|
2
|
160
|
SK & C
|
|
161
|
Lage bedre opplegg for starting av tester fra extensionpoints.
|
|
162
|
|
|
163
|
Christian
|
|
164
|
Fikse property view p� user actions
|
|
165
|
St�tte endring av form element name, id osv.
|
|
166
|
Fikse project property view
|
|
167
|
Vise feilmelding p� sletting av elementer fra side hvor element allerede ligger i en UserAction
|
|
168
|
|
|
169
|
Stein K�re
|
|
170
|
Cubic Runner
|
|
171
|
Params
|
|
172
|
i18n
|
|
173
|
|
|
174
|
Erlend
|
|
175
|
Custom Elements
|
|
176
|
HTML exporter
|
|
177
|
|
|
178
|
|
|
179
|
Espens Cubic Diary (norwegian):
|
|
180
|
------------------------
|
|
181
|
|
|
182
|
Jeg har delt det inn i tre kategorier.
|
|
183
|
Funksjonelt: dette er funksjonalitet som jeg f�ler mangler, jeg ikke forst�r, eller som jeg kunne ha tenkt meg � ha med
|
|
184
|
Visuelt: dette er look and feel
|
|
185
|
Feil/Bugs: ting som jeg mener er feil og eventuelle kr�sj/stacktraces
|
|
186
|
|
|
187
|
|
|
188
|
------------------------------------------------------------------------
|
|
189
|
Funksjonelt:
|
|
190
|
------------------------------------------------------------------------
|
|
191
|
- Det mest kritiske er
|
|
192
|
- � f� inn den XPath st�tten vi snakket om, samt
|
|
193
|
X - at det m� v�re mulig � redigere en User Action.
|
|
194
|
- Uten disse to blir prosessen med � skrive og vedlkeholde testene ekstremt vanskelig og tungvindt.
|
|
195
|
|
|
196
|
- F�lgende burde v�rt bundlet med:
|
|
197
|
- One click instalation med:
|
|
198
|
- Et Cubic prosjekt
|
|
199
|
- En batch fil som starter Jetty/Tomcat med en demo webshop installert som Cubic testen
|
|
200
|
bruker. Da har man noe � se p� slik at man kommer igang lettere
|
|
201
|
X Aspekt, hva er det?
|
|
202
|
- Bytte Add med Check for eller lignende
|
|
203
|
- Hvorfor er det skille mellom Input elements og Controls
|
|
204
|
|
|
205
|
- Vanskelig � forst� hva man skal gj�re n�r man har laget testen. Hvordan bygger/kj�rer jeg
|
|
206
|
testene?
|
|
207
|
- Burde hatt en knapp i Toolbaren med "Generate" eller lignende slik at man kan kj�re
|
|
208
|
Cubic testene.
|
|
209
|
- Burde hatt muligheten til � velge Cubic under "Run as", slik at du ikke m� inn under
|
|
210
|
den Cubic test greiene for � kj�re testen
|
|
211
|
- Hvordan kan jeg sjekke p� andre elementer enn de som er i toolboxen? f.eks Table, div, etc
|
|
212
|
- Page vs State, burde v�re en av delene eller to ulike
|
|
213
|
- "Description" feltet p� "New Cubic Test" burde wrape linje n�r man kommer til slutten av
|
|
214
|
boksen. Brukeren burde ikke m�tte trykke enter i feltet slik man m� n�.
|
|
215
|
X- Select box burde hete Select list ettersom det er hva den heter
|
|
216
|
- Extension Point??? Hva er dette?
|
|
217
|
X- I Package Explorer burde man n�r man h�yre-klikker "New" muligheten til � legge til en
|
|
218
|
Folder ogs�, uten � m�tte g� til "Other". Foldere vil man bruker til � strukturere
|
|
219
|
testene.
|
|
220
|
- "Custom action": Enten burde man kunne definere sine egne Actions (det hadde egentlig v�rt det beste) slik at man
|
|
221
|
f.eks kan teste p� back knappen eller andre ting som man kan gj�re vha Javascript. � lage det som en custom action gj�r
|
|
222
|
at man vil kunne skrive denne slik man selv vil.
|
|
223
|
- N�r man skriver en test s� burde man kunne scrolle "uendelig". Slik det er n� s� m� man legge en kontroll langt nede p� siden
|
|
224
|
for at man skal f� mere plass. Her burde man kunne bare scrolle videre nedover ogs� kunne man legge til noe der. Kanskje
|
|
225
|
ikke uendelig, men ihvertfall burde man sette av litt mere plass. Burde ogs� v�re en opsjon og scrolle horizontalt, ettersom
|
|
226
|
det kan v�re brukere som �nsker dette ogs�. Man leser bedre bortover enn nedover.
|
|
227
|
- I18n and Parameters burde hete Locale, Regional Settings eller noe lignende.
|
|
228
|
- Arv av Aspects, burde kunne koble en aspect til en annen dersom man �nsker det i enkelte tilfeller. Da ville man enklere
|
|
229
|
gjenbruke Aspecter p� tvers av tester... muligens er dette ikke smart men...
|
|
230
|
|
|
231
|
------------------------------------------------------------------------
|
|
232
|
Visuelt:
|
|
233
|
------------------------------------------------------------------------
|
|
234
|
- Sidene man drar ut burde ha shading og i tillegg burde "tittel feltet" v�rt st�rre,
|
|
235
|
f�lge boksen og en st�rre font. Burde ikke trime strengen, dette burde brukerne besteme
|
|
236
|
selv
|
|
237
|
Startpage:
|
|
238
|
- Startpage er veldig uklart fremstilt, burde ogs� v�re alignet midt p� og ikke oppe i
|
|
239
|
hj�rnet n�r man starter
|
|
240
|
- Burde gjemme URL'en slik at den vises bare i f.eks properties view'et
|
|
241
|
- Burde hatt et annet icon for start, en running eller noe slik man bruker n�r man tegner prosessflyt
|
|
242
|
- Toolbox panet er for stort, det trenger ikke v�re st�rre enn den lengste teksten
|
|
243
|
- Aspect og Page/State burde ha ulike bokser
|
|
244
|
- Aspects burde komponeres inn i bokser eller lignende. Slik det er n� blir det veldig
|
|
245
|
rotete skjema dersom man bruker et aspekt som g�r p� f.eks 5-6 sider/tilstander
|
|
246
|
- St�rre font, alt er veldig sm�tt og puslete slik det er n�.
|
|
247
|
- Burde vist testene p� en bedre m�te i "Package Explorer"
|
|
248
|
- Alle iconer burde v�rt st�rre idet man trekker de inn i en Page/State/Aspect. Da ville det
|
|
249
|
v�rt enklere � f� oversikt. N� m� man myse veldig f�r man ser hva det er.
|
|
250
|
- Iconene burde i st�rre grad lign elementene de skal teste, slik som knapper.
|
|
251
|
Muligens burde man sett p� hva Web Tools bruker av iconer.
|
|
252
|
|
|
253
|
------------------------------------------------------------------------
|
|
254
|
Feil/Bugs:
|
|
255
|
------------------------------------------------------------------------
|
|
256
|
X Kan ikke redigere en User Action. Properties er tomt og man kan ikke redigere.
|
|
257
|
X- Undo p� en Page/State med en selectbox f�rer til Exception
|
|
258
|
- Tab order p� New Cubic Test User Input er ikke riktig, burde kunne Tab'e seg fra element til action og til input.
|
|
259
|
N� s� g�r man fra element til "Add New User Input"
|
|
260
|
- Burde legge p� Keyboard shortcuts p� _alle_ knapper i alle Wizards
|
|
261
|
- Properties view'et. F.eks n�r man har lagt til en kontroll og vil endre f.eks Identifier Type sin value fra Name til ID,
|
|
262
|
s� burde man kunne skrive med keyboardet ID ikke m�tte bruke pilene.
|
|
263
|
- Har du en TXT inne i en Page/State som er lengre enn Page/State boksen s� vises ikke hele teksten n�r man markerer TXT kontrollen.
|
|
264
|
Den ser ut til � bli liggende bak eller noe slik.
|
|
265
|
- Fungerer ikke � generere Watir tester for kataloger, f�r f�lgende melding i Consolet:
|
|
266
|
org.eclipse.core.internal.resources.ResourceException: Resource /PBA-Fondshandel/dev/runtime-EclipseApplication/PBA-Fondshandel/tests does not exist.
|
|
267
|
- N�r jeg cut'n paster en Aspect fra en test inn til en annen s� kan jeg ikke sette p� Connectors fra Aspect'en til en Page/State
|
