There are three classes of requirement; author, student, and maintenance. The first two of these (as defined by the company) are dealt with in this chapter. Maintenance requirements are set out in the Design and Maintenance section, chapter 6.
This chapter has a cyclic structure which follows the development of the author language functions from their conception to finalised form. Initially there is an overview of the kind of functions originally required by the company. Next comes a re-assessment of the overall requirements, and a detailed (idealised) requirements list for the match functions. Following on from this, section 3.3 proposes a number of possibilities for match functions, and the reasons for their incorporation into (or rejection from) the finalised match functions. Finally, in section 3.4, the requirements for an intelligent spelling assistant (proposed in 3.3) are developed.
Chapter two has already described the kinds of questions that are used in CBT and other training/testing situations, and what is involved when an instructor analyses a student's response. A synthesis of all the foregoing resulted in the Requirements Document: a practical approach to answer analysis in a commercial environment (Chapter 4A).
Note that apart from the match function requirements, I was given free rein to design the other functions as I thought fit, using the original requirements list (appendix A) as a basis for design. Also, the term "minor spelling errors" was not defined by the company; this led to my being able to pursue the topic (section 3.4) in greater depth than would have been necessary had they defined it.
The initial project specification supplied by the company (appendix A) called for the development of an AL which could be used for the production of CBT software. After discussions with the company's project supervisor (Mr C Dean), and with authors who use commercially produced authoring languages (and a perusal of the ALs available to them [CAI 1988, Compsoft 1987, CSR 1985, FPC 1985, FPC 1987, IBM 1987, Learncom 1986]), I identified four basic categories of functions which are necessary to do this:
For each of these groups I have outlined some of the more obvious functions, in order to set the scene. Please note that these do not comprise a final list; they are not to be taken as a specification of all the functions that are needed in an Authoring Language, nor are they necessarily in the best format. At this stage I merely wish to give examples of the kind of functions required, and refinements will come later.
Briefly, these are functions that compare a student's answer with a model answer. Usually an author can specify whether such things as capital letters, punctuation, or spaces are criteria that should be considered by the function. For example, if case is important, then a function might accept 'Mr Smith' and 'MR SMITH' as valid answers, but reject 'mr smith'.
exact-match(model, answer)
If the answer matches the model exactly, the function returns TRUE.
match(model, answer, match_parameters)
If the answer matches the model exactly, the function returns TRUE. Match parameters are used to indicate whether case, punctuation, or spaces are important.
mc_match(model, answer)
This is a Multiple-Choice matching function. The model is a letter or number, and the answer must match exactly for the function to return TRUE.
find-word(model, answer)
The model in this case is one word. If it occurs in the answer the function returns TRUE.
ordered-search(model, exclusions, n, answer)
The model consists of a number of words, n is a number which indicates how many of these must be in the answer in the order given for TRUE to be returned. If any exclusion is found, the match fails (ie, FALSE is returned).
any-search(model, exclusions, n, answer)
This is similar to ordered_search, except that the order of words in the answer is unimportant.
number-match(model, range, answer)
The model contains a number (real or integer), range specifies the allowed variation in the answer (either as a percentage, or absolutely). TRUE is returned if a number that fills these requirements is found in the answer.
clearall
Clears screen or current window.
clear(x,y)
As above, but between rows x and y inclusive.
place(text)
Print text string at cursor position. Text could be anything from one character to a screenful.
putmessage(n)
n is a message number, which refers to standard (user changeable) messages.
cursor(x,y)
Locates cursor at row x, column y.
hidecursor
Hides cursor.
showcursor
Reveals cursor.
make-window(n, rowl, coll, row2, col2, setup parameters)
n is the window number, rowl and coll identify the top left corner, row2 and col2 the bottom right. Setup parameters could include frame on/off, and the background colour.
select-window(n)
Selects the chosen window as the area where all screen addressing operations take place.
delete-window(n)
Deletes window numbered n.
The default is to have no windows, all screen addressing commands would refer to a standard screen (probably either 80x24 or 80x25, depending on the implementation).
delay(n)
Causes a delay of n seconds (or other suitable time unit).
set-match-default(match type)
Sets up default values for the matching functions, so the author need not specify the type each time a match function is used.
chances(n)
n specifies the number of attempts that may be made for a given question. If this function is not used, then the default is one.
timeout(n)
n specifies the number of seconds within which a reply must be made. If this function is not used, then there is no timeout.
skip
Allows the student to skip an answer. (This could also be set as a parameter in the match functions).
goback(n)
Goes back n questions.
Program control will also need some form of IF THEN ELSE structure, CASE, WHILE, and possibly others.
Some form of initialisation procedure is desirable, so that the program is able to function on a variety of machines (eg 24 or 25 line monitor, LCD screen or VGA monitor). This procedure need only be run once for each machine; setup parameters are written to disk.
getanswer
Reads an answer from the keyboard.
keepscore
Keep score if desired, this could be a directive at the head of the program.
add(n)
Adds n points to the student's score.
delete(n)
Deletes n points from the students' score (eg for an incorrect response).
writescore
Writes the score at the current cursor location.
listwrong
This is a directive at the head of the program which causes the question numbers/names of incorrectly answered questions to be written to a disk file, for later analysis.
In these examples, no consideration has been given to the Human-Computer Interface (HCI); interaction could be through a menu, a mouse, a lightpen, or a touch sensitive screen might be used. Obviously, when a student inputs a long answer, the only practical method is to use the keyboard (the future may bring the possibility of voice activation).
An advanced system would have many additional features, such as those listed below, all of which, except the last, are present in the ALs that I looked at (unfortunately not all in the same package). For the author to take full advantage of these facilities there should be a powerful text and graphics editor, and advanced debugging facilities.
Registration of learners, and the enforcement of security procedures. This includes allowing the author to keep track of a number of different students who might work through only part of a CBT package on one occasion, and return at another time.
A database of learner input (eg a question bank), and a random question selection facility. This could include a library facility for pictures/graphics. Eg, screensful of information could be stored, and retrieved using keywords, and then incorporated into a CBT package.
Interfaces with video and audio equipment, digitised sound and graphics/pictures, animation.
Students' answers (rather than just results) could be saved, for future analysis by a teacher/trainer. (In certain cases the rulings of the Data Protection Act may influence what can be done).
Ability to run other programs and routines from within the CBT package, as well as being able to write subroutines in the AL.
Functions which compile statistics on all aspects of student performance, and which generate reports.
There could be provision for the student to have a printout of questions answered, and the program's responses, plus help and suggestions about areas of weakness.
Considering the limited time available for this project, and its overall complexity as indicated by the foregoing examples of author language routines, by agreement with the company's project supervisor the range of the initial specification was reduced to a more manageable level (ie one in which all or most of the requirements could be met). I decided to concentrate on producing the match functions, and a minimal number of functions which would be required to run these. The reason for this was because one of the most important aspects of an AL is its matching functions. Without these, even the most visually attractive and structurally complex in terms of branching ability, remedial loops, help options, etc rapidly lose both their appeal to students, and their pedagogical usefulness, and it is in this area that commercial ALs show the greatest limitations.
These are the requirements that the company specified should be in the match functions.
The type of match (eg exact, ignore punctuation, ignore case, exclude a word, error allowed in number match, etc) for each match should be specified by the author. If all, or most, of the matches in a program are to be the same, then defaults could be set at the top of the program, to reduce the workload of the author. This would also limit the likelihood of errors being made by the author when specifying the match type.
There should be some form of feedback for wrong answers, as well as some method of dealing with expected wrong answers.
There should be a sample answer option, so that the student can see the expected format of the answer. Alternatively, there could be a message to tell the student the type of answer that is expected.
The author should be able to specify any n from m options to be matched.
The match procedures should indicate author errors (eg unknown match parameter supplied) by means of suitable messages written to the screen.
General requirements, plus:
Various types of match should be catered for: exact search
for one or more words (either ordered, or in any order), exclusions, acceptance of spelling errors, ignore case, spaces, punctuation. The author should be able to specify which combination of these s/he wants.
Wildcard characters should be allowed in the model answer.
Ideally, there would be no maximum length of response. In fact, it should be possible to match two or more sentences, or even a whole paragraph.
These include general match requirements, plus:
The ability to match any kind of number (positive or negative, integer or real, fractions).
The type of match possible should include exact, number of decimal places, range, percentage, and possibly words or figures.
Punctuation should not be important, unless the author wishes to make a distinction between real and integer numbers, (eg 23,000 and 23 000 and 23000 and 23,000.0 are equivalent), whereupon there should be a message generated by the function to tell the student the type of answer expected (eg real, integer, fraction), unless this is part of the test, in which case the author should be able to suppress the message.
There should be some provision for standard abbreviations.
International use should be catered for (if desired by the author) , so that a comma can be used instead of a decimal point. A message to this effect should be shown.
If using words, the author should be able to specify the range by means of some suitable notation, for example, as ONE TO TEN, or as in Pascal, ONE..TEN (assuming an enumerated type, say small-numbers, had been previously defined).
Provision of currency and other symbols (eg Cr, Debit) is required, the exact details depending on the application domain.
There should be the provision to input a mathematical formula as part of a question, with a random number generator producing coefficients, so that the student can be presented with a series of mathematical questions, without the author having to write many similar questions.
In this section the major ideas that occurred to me for the production of matching functions are described, along with the reasons for their exclusion or inclusion in the final product design. For the reader to appreciate the evolution of the match functions, these ideas have been listed in the order that they were investigated.
The match function could take a model answer, exclusions (optional), a student's answer, and a list of parameters to control the type of match required. It could return a list indicating the results of the match.
match(model, exclusions, answer, match_params) --> results
This is the model answer, the format is flexible. It could be a sentence with full punctuation, a list of words, or even just one character (for mc matching).
This is an optional parameter, which indicates words or phrases that are not wanted, and which will cause match to reject an answer (by return of suitable values in results list). Its omission could be indicated by inclusion of an extra comma in the variable list.
A list of characters and/or numbers indicating the type of match required. Match types could be as follows (the numbers are for reference only, the final product might use different ones):
Obviously, for example, 1 and 5 appear to be mutually exclusive; so if the author tries to use this combination, there should be an error message when the program is run, and the match function should terminate.
Multiple choice questions could be answered by combining options 5 and 6.
eg: match("A",,answer,5,6) --> results
means that the function match requires the answer to be either "A" or "a". Two commas indicate omission of the exclusions list.
eg: match("A B",,answer,5,6) --> results
means that either A or B is acceptable.
Blank fill questions could be answered using options 1 and 3, combined with option 5 if case is not important,
eg: match("hello",,answer,1,3,5) --> results
There would have to be provision for a variable number (including zero) of match parameters to be used, and for the author's convenience, the order of these should not need to be fixed. The use of commas to indicate omitted variables is not user (ie author) friendly, and could be a source of errors. Implementation problems that could arise are to do with the match parameters; some form of list handling mechanism would probably be the most practical implementation to adopt. After consultation with the company's project supervisor, the option of having an exclusions list contained within a function was discarded as being too involved; if an author wishes to check answers for 'exclusions' s/he can simply call the function twice, the first time to check that the excluded words do not occur in an answer.
In the discussion above, I have suggested that an exact match, and a match where case was considered unimportant, are mutually exclusive. Really, this is not the case. We have to take into account the fact that when we use the word "match", in a computing environment we generally think of "syntactic match". For example, we usually say that two character strings match if they have the same sequence of characters; they do not match if one string is shorter than the other, or the nth characters are different. However, this clouds the issue when we want to match a student's answer to a model answer. On most occasions what we want is the meaning of the answer, not its structure.
The chart below shows how match types may be separated into syntactic and semantic groups.
SYNTAX SEMANTICS
exact -----> exact
punctuation one word (phrase)
spaces 2 or more words (fixed order)
2 or more words (any order)
spelling <---> verb endings/tenses/plurals
<---------------------->
part of word
case
As the arrows indicate, syntax and semantics are not independent. An exact syntactic match implies a semantic match (at least as far as the program is concerned - it is possible that the student has not understood his/her answer, but that is another matter). Spelling obviously can alter the meaning of words, as well as changing the meaning completely. Not so immediately apparent is the fact that all the other items can affect the meaning of a word or phrase. Some examples are shown below:
Punctuation (eg defining and non-defining relative clauses [Quirk et al 1984])
Compare "My brother who lives in York is visiting me next week" and "My brother, who lives in York, is visiting me next week".
Case
Compare "My bill's arrived" (electricity, say), and "My Bill's arrived" (spouse).
Spaces
Compare 'black bird', and 'blackbird'.
We now appear to be in rather a paradoxical situation; on one hand it seems that the separation of match types into two classes is necessary (eg we may want an exact semantic match, but not care about spelling errors), and yet on the other hand, we can see that syntax can have a marked affect on semantics.
However, we are now in the realms of natural language understanding. Rather than simply trying to match a few key words, we are looking for a detailed analysis of complex language constructs. Although much work has been done,' and is being done, in the field of natural language processing, I do not intend to discuss this topic, as it is outside the scope of this project. Suffice to say, there is as yet no way around the problem of language analysis, as far as this project goes. However, this does not mean that we should abandon the idea of separating the match types - it is a useful distinction to make, and brings us at least part of the way to our ideal match function.
This leads to the following possible match functions.
match(semantic-type, syntax_type, model, answer) --> results
examples:
match(multiple_choice, model, answer) --> result
No syntax parameter is needed, the result is either TRUE or FALSE.
match(exact, syntax_type, model, answer) --> result
The result is either TRUE or FALSE.
match(ordered, syntax-type, model, answer) --> result_list
The result_list is list of booleans corresponding to words in the model.
eg
if model = cat sat mat
and answer = dog sat mat
then result-list = f t t
match(anyorder, syn_type, model, answer) --> result_list
The result list is list of booleans corresponding to words in the model.
eg
if model = cat dog mouse
and answer = dog rabbit cat
result-list = t t f
How are the syntax type match parameters to be indicated? Possible methods are by using a list of characters/numbers, or a powers of 2 method, in which each permutation can be indicated by a unique integer. Syntax_type parameters could be, for example:
E exact match required
S spaces important
P punctuation important
However, as has been mentioned in the Review (chapter 2) there is a major difficulty with returning a detailed analysis of an answer, namely the combinatorial explosion of responding to an analysis. It would be extremely difficult to organise program branching on the basis of a results list as shown above. At most, we could categorise the results into numbers of correct words/phrases (ie have some kind of graded analysis, so that totally wrong answers are dealt with differently from partially correct answers), and branch on this basis. For the purposes of informing a student which parts of an answer were acceptable, it is probably not worth the time and effort of an author. The simple solution of saying "you found five out of seven", followed by a list of correct answers, is adequate. No doubt, this is one of the reasons why commercial functions do not have this kind of facility.
To aid the author, it is better to give the match functions different names, eg multiple_choice_match, exact_match, etc, thus separating the semantic and syntactic aspects still further; semantics equates to the function name, syntactic requirements are in the match type parameters list.
A separate function to set the model answer, so that setting and matching are separated, would be convenient when dealing with many model answers in one function, or with long model answers (details below). In addition, setting questions could be separated from the subsequent analysis of responses.
set-answer(label, model_answer)
The label is a variable name, and is used as a parameter in the match functions.
eg set_answer(l,The cat sat on the mat)
This function was not developed due to limitations of time; my effort was concentrated on producing the match functions. Note however, that this function is not necessary with the implementation language used; similar results can be obtained by using the #define construct. Refer to 'Defining Models' in the User Manual (section 4.6) for details.
The match functions can include options for displaying sample answers, help, and messages. Message parameters are optional in all cases. Any combination may be chosen; if none are selected, then the default will be used. This default may be set by the author at the start of the program, using a set_message_params function. if this is not done, then the default is to NOT give Help, or Correction messages, but the sample answer will be available to the student.
Example:
match(multiple-choice, model, message_params, answer)
A = sample Answer, so the student can see the expected format. In order to save authors' time, these could be pre-defined, since only the content would vary, not the general method of use.
C = Correction message to the student saying that the answer has been accepted, although there are minor errors.
H = Help/hint - if this is used, then an input parameter (containing the message) must be used.
If A or H are used by the author, the student can use the facilities by selection of a function key.
The company's project supervisor thought that this type of facility was not necessary, and so the ideas were not expanded.
For convenience, the set answer function, described above, has been used as a context within which to discuss this, but the same considerations apply even if this function is not used. For model answers where a number of options are acceptable, the following format might be appropriate (the backslash as a delimiter between options is only an example, it might be any suitable character).
set_answer(l,The cat sat on the mat\The cat is sitting on the mat\The cat is on the mat)
There is a lot of repetition here, and one method of overcoming this and a similar problem associated with the acceptance of synonyms, would be to allow options within the model answer. An example (incorporating BNF notation) portrays this:
set-answer(label, [The|A] cat [sat|is [sitting]] on [a|the] mat|rug]).
But this is far too involved for practical purposes. An author would need to spend much time in ascertaining that the model answer was corrects and so this method is highly inefficient in terms of author time. Something much simpler is needed, and so this approach was not pursued in the development of the functions.
A problem arises with the analysis of multiple answers for one question. For example, if a model answer were, "Valve A controls function X, and valve B controls Y", an author would most probably want to accept the clauses in any orders although within clauses the order of key words is important (ie function X does not control valve A). one solution to this is to have nested matches, with a different match type for each group.
match( ... (A controls X), (B controls Y) ...
-------------- -------------
fixed fixed
------------------------------
any order
However, this is leading away from simplicity, and is not likely to enhance a suite of match functions to any great extent, and therefore this option was not pursued.
We want to be able to accept minor spelling errors, so in addition to using wild characters in the model answer, a spelling checker would be most useful. However, there is a major problem - a dumb spelling checker, of the type that is used in word processing packages is worse than useless (I say "worse than", because all it would do is waste processing time!). What is needed is an intelligent spelling assistant. No further amplification is given here, as this idea is developed fully in section 3.4.
Augmented Transition Networks (ATNs) are similar in concept to Finite State Machines (FSMs). There is a set of allowed states, a starting state, one or more completion states, and a set of transitions.
Although grammatical constructions are theoretically limitless, in practice they are not. There is a limited lexicon in general use, sentences that contain many clauses are difficult to understand, and are not common, and inter- sentential reference has a limited range. If an arbitrary limit is placed on the constructions that we intend to deal with, then, in theory at least (because the system we want to analyse may still be too large for us to cope with), we can accept Sampson's (1986) statement that 'it is mathematically demonstrable that any grammatical phenomena which are clearcut enough to be defined in any format whatsoever are capable of being stated in terms of ATNs', and thus ATNs can be useful in certain natural-language processing systems.
The LUNAR project is an example of a large ATN system doing just this; at a geological congress in 1971 scientists asked questions about lunar rock. There was no guidance about how they should phrase their questions, and yet most of their queries were understood and answered correctly.
Sampson has produced a small system using ATNs which is used to help children learn to read clocks, and which has proved most successful.
In both these examples the domain is restricted; since the company is looking for a general matching system, the use of ATNs is not appropriate. However, their use should not be totally ruled out. Future development using a restricted technical vocabulary and limited sentence constructions is one possibility.
In conclusion, rather than trying to tackle all possible problems, it is far better to restrict the scope of the match functions, and keep them simple for the author to use. By trying to do too much, we will end up with an overly complex system which is difficult to use, and hence will not be used correctly, or to its full potential (which is wasteful of development time). Chapter 4 proposes a set of simple, elegant, and powerful match functions (and associated functions necessary to use these) which incorporate most of the requirements set out, but which avoid the complexities discussed above.
The matching functions should be able to deal with minor spelling errors (unless the author wishes to set a spelling test). The reader may be considering the use of a spelling checker such as is found in word-processing packages. The use of one of these, however, is not appropriate, since all it would do is work through a list of words, to try to find an exact character match for the answer. If the answer could not be found, then some message indicating that an error had been made (possibly also with some alternatives to the misspelt word) would be output. What is needed is some mechanism whereby the user's answer (from now on called answer) and the model answer (from now on called model) are related. This would take a model and an answer, and by the application of some rules, would determine what kind of error (if any) had been made, and whether the answer were acceptable; in other words an intelligent spelling assistant. So, this is not simply a case of preferring to use an intelligent system rather than brute force; brute force won't work.
In order to determine the rules to be applied, we need to know the potential errors that could be made, and this is the subject of the next section. Following this, 3.4.2 looks at the frequency of keyboard errors. Some of the difficulties that arise in a complete system are discussed in 3.4.3, and finally (3.4.4) there are the requirements for the spelling assistant as it is to be in the match functions.
I have identified two kinds of error that may be made, namely keyboard errors (caused by poor typing ability), and true spelling errors, where the writer is unsure of, or knows incorrectly, the spelling of a word. Each of these will be dealt with in turn.
Gentner et al [1983] have classified keyboard errors as follows:
Transposition - the interchange of two consecutive letters, eg teh for the.
interchange across n letters - the interchange of two non- consecutive letters, with n letters intervening (n > 0). eg jamor for major.
Migration across n letters - one letter moves to a new position, with n letters intervening between its correct and its new position (n > 1). eg atht for that (n = 2).
omission eg omt for omit.
insertion eg asnd for and.
Substitution - when a wrong letter is typed in place of a correct letter.
Doubling - when a word containing a repeated letter is typed so that the wrong letter is doubled. eg bokk for book.
Alternation - when two letters alternate, but the wrong sequence is produced. eg thses for these.
The kinds of errors that are likely to be made are similar in effect to some of those described above. A few examples are shown below. Section 3.4.3 explains why even though the end results overlap, there is a case for separating the causes of errors.
Transposition. Transpositions of letters in words can often happen, particularly with ie/ei.
Omission. The word beginning, might easily be mis-spelt as 'begining'. Consider also the American spelling of colour, (ie color), which most times an author would probably wish to accept.
Insertion. eg chiming, might be mis-spelt 'chimeing'.
Substitution. Often, there is more than one acceptable way of spelling a word. An example is the word 'analyse', which may also be written 'analyze'. Homophones can also cause this, for example 'dependant' and 'dependent'.
Interchange, migration, doubling, and alternation errors as defined in the previous section are not likely to arise from purely a lack of spelling ability on the user's part.
Phonetic spelling problems. A similar problem to that of homophones arises with the use of interactive video or audio facilities, where names may be heard, but never seen. eg is the doctor's name Brown, or Browne, or Braun?
Other well known difficulties that might arise are with, for example, wr --> r, ough --> ow, ough --> uff, etc.
The inter/intra syllabic recognition phonetic match anomaly (or 'effing' problem), poses an interesting problem. Consider the spelling of phenolphthalein, where an author might wish to accept a spelling which contains the letter If' in place of I ph, . It is not sufficient merely to have some rule in the spelling assistant stating that phonetically 'f' is equivalent to 'ph' (as does TopClass [FPC 1987]). Within a given syllable this is true, but the simple rule falls down with a word such as 'haphazard', where the letters p, and h, do not occur in the same syllable. Therefore, a syllable analyser appears to be a necessary part of a fully functioning intelligent spelling assistant.
According to Grudin [1983] although novices make more typographical errors than experts, their errors are orderly (!). Most of these are substitution errors, and many of the miscellaneous ("other") errors are words in which two substitutions appear to have been made. The following chart, adapted from Grudin, sums the situation.
| Typist | Speed (wpm) | Error rate (%) | ERROR TYPE (% of all errors) | ||||
|---|---|---|---|---|---|---|---|
| Insertion | Omission | Substitution | Transposition | Other | |||
| expert | 75 | 1.0 | 43 | 14 | 27 | 7 | 14 |
| novices | 20 | 3.2 | 9 | 4 | 75 | 4 | 11 |
Substitution, insertion, and omission errors refer to single letter errors in otherwise correctly typed words.
These results have been determined using transcription typing, so although they most probably are a good indication of the areas that need to be focussed on, their absolute values may be in error with respect to free typing (though the typist most likely has a mental model to copy - I do). Additionally, note that since only eight students and six experts took part, we cannot assign any statistical validity to these results, but they do give a gross indication of the situation.
From Grudin's work, we can see that the keyboard errors made by typists depend upon their ability; this leads to the conclusion that perhaps there should be two user models (ie expert and novice) in the spelling assistant , so that it could search for likely errors before unlikely ones. It is, as others have said, a case of selection versus calculation.
From the previous two sections we might conclude that an intelligent spelling assistant ought to contain the following:
Two user models (novice and expert).
A syllable analyser (for the 'ph' problem).
Keyboard error rules (transposition, interchange, migration, omission, insertion, substitution, doubling).
Language dependent error rules and phonetic analyser, for ph --> f, s --> z, y --> ie, etc.
However, in practice, all these features are not necessary, and might even lead to a complicated and slow system which could be an irritation to the end user.
A little thought shows that having two user models is impractical and unnecessary. Impractical, because prior to using the system with the match functions, the system would need to know which model to use. This could only be done by someone (probably the end user) telling the spelling assistant (after a prompt) the user model required, but how is the user to know his/her level of typing competence? As an aside, in a fully integrated CBT system, the user models could be incorporated into student records in which the parameters would need to be entered only once. The level of typing competence could even be assessed automatically by the provision of a computerised test.
The use of two user models for typing competence is unnecessary, because the keyboard error rules for both models are the same, only the selection of them would occur in different orders if we took a selection versus calculation' approach. However, since the rules are extremely simple, there is no necessity to select them in any particular sequence before using them. Furthermore, with hindsight, from an implementation standpoint selection before use would require far more computing power than simply taking the rules in a fixed order.
As far as developing an intelligent spelling assistant is concerned, there is no 'ph' problem, and a syllable analyser is not needed. A user is hardly likely to type f in place of ph, if the ph does not sound like f! Therefore, in practical terms, there is not much point in going to the trouble of finding out whether a particular combination of the letters p and h is within or between syllables.
Language dependent error rules should be separate from the keyboard error rules, because languages differ. So, in English, the substitution of the letter f for ph would be acceptable (ignoring for the moment the 'ph' problem) to the phonetic analyser, whereas in Turkish it would not, since the juxtaposition of the letters p and h never results in the sound /f/. A few examples of what the language rules for English should cope with are listed here: ph <-> f, ck <-> k, wh <-> w, wr <-> r, z <-> s, ough <-> uff, ough <-> ow, ough <-> u. (This last example might arise if speakers of American English were to use a system with rules for British English; their version of 'through' is 'thru'.)
Rapid prototyping of a simple intelligent spelling assistant (refer to program listings) has revealed that to use a recursive system would lead to wrong or nonsensical answers being accepted. There must be a limit to the number of allowed errors in each word, but this limitation itself poses problems. Allowing, say, one error in each word is a simple, but rather unsatisfactory approach, because although in a short word this is acceptable, in a long word it would be more appropriate to accept more than one error. A possible method of overcoming this is to allow a variable number of errors, depending upon the length of the word being analysed (say one error for each six letters).
So far, no consideration has been given to how the two distinct sets of rules discussed previously might be used in practice; this is the topic for the remainder of the chapter.
Consider first a system that has a keyboard error assistant which deals with the keyboard errors described in section 3.4.1., and which allows one error per word. As an example, let a model be the word 'analyse', which has an alternative spelling of 'analyze'. If an answer were given as 'naalyse' the spelling assistant would accept it as the word 'analyse' which happens to contain a transposition error. If an answer is given as 'analyze', the assistant would accept it as 'analyse' with a substitution error, even if there had been no keyboard error on the part of the student (s/he might prefer the spelling 'analyze'). Now, if the answer were given as 'naalyze', it would be rejected, even though there is only one keyboard error (an --> na).
If keyboard errors and language dependent errors were separate, so that there were rules for both, the last example would be accepted as the word analyse, which happens to contain one keyboard error, and one spelling error (although in this case it strictly isn't an error, but merely an option). Consider an imaginary system that contains in addition to the keyboard rules, a spelling rule which allows the transition s --> z. Now, if the answer were given as 'naalyze', the keyboard rules would deal with an --> na, and the spelling rule with s --> z. If an answer were given as 'naalyte', it would be rejected, because there is no spelling rule which allows the transition s --> t (so the system sees two keyboard errors, but we only allow one).
Great care would have to be taken over the selection of language dependent spelling rules, they would probably have to be severely limited in number, and they should not interact. A small example illustrates this. Let the system contain rules that allow s --> z, x --> ks, ch --> teh. Let the model be axe, and a nonsense answer 'atchz'. Using these rules, and allowing interaction, the answer would be accepted (atchz --> achz --> achs --> aks --> axe).
In practice, the system might use the keyboard error rules first, since keyboard errors are more likely to occur than true spelling errors, and also many spelling errors are covered by keyboard error rules. If a language dependent error were dealt with using a keyboard rule, so be it. Only if an answer could not be judged as acceptable would the language dependent rules be used.
Assuming that the answer could not be 'understood' using just the keyboard rules, the following operations could take place. At each occurrence of an error (ie character mismatch) the rules would be checked to determine whether that part of the answer corresponds to the related part of the model. If so, the analysis would continue, but only keyboard errors would be considered (since only one spelling error is allowed in a word). If the error could not be analysed using the language rules, then the (more general) keyboard rules could be used to determine the relationship between model and answer. For the remainder of the analysis, only the language dependent rules would be used, since only one keyboard error is allowed in each word.
The assistant should be able to deal with transposition, omission, insertion, and substitution errors. It should allow just one error in any word. These four categories cover the majority of typographic errors (refer to table 1), and most of the cases of spelling errors. Additionally, from personal experience, I know that the typing errors I make (and this goes for colleagues also) far outweigh the spelling errors that I commit, and so if a model based solely on typing errors is developed, it should be adequate, even if it is not quite optimal.
The spelling assistant should work in conjunction with models that contain wildcards and wildcharacters, of which there may be any number in a word. A word is defined as any group of characters that does not contain a space character, in other words, it is what we consider a 'word' to be in everyday speech. A wildcharacter in a model represents any single character, and a wildcard represents an unlimited (within the constraints set by the maximum answer length) number of characters, including zero. Within each group of characters that is delimited by a wildcharacter/card only one error is allowed.
If time permits, development of a spelling assistant incorporating language dependent match rules will take place.
Preface | Contents | 1 Introduction | 2 Review | 3 Req. analysis | 4 Req. documents | 5 Specification | 6 Design | 7 Verification | 8 Discussion | 9 PAL manual | Appendix A | Appendix B | Appendix C | Glossary | References | Index