This concluding chapter is divided into four sections. Section one is devoted to a discussion of the problems encountered after the functions had been implemented (implementation problems themselves are discussed in chapter 6.3). Some possible solutions are also covered. Following this, the company's comments, and new requirements which have come to light through this project, are discussed. Also, some ideas for the long-term future are considered. Section 8.3 contains my recommendations to the company regarding future work on the author language functions, and finally section 8.4 deals with the lessons that have been learnt by carrying out this project.
If a function raises an author error, and if that function is contained within a program loop, then control will not pass out of the loop. Control+C must be pressed to escape, and the program will abort. This problem will not arise when the author error functions are fully implemented according to the specification. One way round this problem is to let an author have access to the value returned when an error occurs (ie AE), but this is messy, and if the author's code is correct, should not be necessary.
message is unavailable to an author because at all times (as far as a user is concerned) the program is waiting for a response (either an answer, or a keypress). There is a standard message with get_answer, mc, and pause, so if an author uses this message function, the message is immediately overwritten. This problem was not discovered until integrated testing of the modules took place.
One way round this problem is to have a user message area and a system message area. This might, however, be a little confusing for users.
show match_params is a function that was developed during the prototyping of the system. It shows, in the header area, the current match parameters. At one stage, although not requested, it was to be included with the other functions, as a bonus to the company. However, since the input and the analysis of an answer are separate, and since analysis must occur after an answer has been obtained, this function cannot be incorporated into either the getanswer function' or the model/answer comparison functions, because when an answer is input the system does not know what the match parameters will be at the time of the match. There is no point in showing the parameters during or after the answer has been analysed.
One solution to this would be to incorporate a parameter list with the function getanswer, but great care would have to be taken to ensure that what we say the parameters are, and what we set them to in the comparison functions are in fact the same. This is an inherently unsafe approach.
An alternative solution would be to have a parameter list only in the getanswer function, but not in any of the matching functions. The current match parameters would then be set at the time that an answer is input from the key- board. However, this then limits the potential power of the matching functions; for any one answer, if more than one comparison with a model were to be done, all the comparisons would have to use the same match parameters. Of course, the set-match_params function could be used to change this, but then we would be back to the problem of how to tell a user what is expected.
There does not seem to be any way round the difficulties mentioned. Perhaps the most suitable answer is simply not to indicate the match parameters that the spelling assistant will be using. Students do not need to know this information, and anyway, they ought to be careful with spelling at all times!
This section considers some improvements that could be made to the author language functions. First there are those that the company has specifically requested, in the light of the development of the project, and then there are those that I consider to be advantageous. Modifications could be incorporated into future versions of PAL with ease, due to the modular approach taken in design and implementation. The reason that this has not already been done is simply lack of time. The final part of this section is of a more speculative nature, and should be considered as a 'sounding board', rather than as a fully developed project plan. Considerable work would have to be done on the feasibility of these ideas, before they could be implemented.
multiple choice A request from the company was that the highlight bar should start on the first non-space character of each option. Multiple line options would be an advantage. The starting column for the stem and options should be author specified.
skip( n ) is a function which will move the cursor n lines down the screen, and relocate the column to 1.
It was suggested that the layout of some of the messages could be improved, but this was not amplified.
With the function pause there should be the provision for an author defined message to complement the standard message. One possibility is to have a function with a different name, where the user specifies the whole message. Another option would be to have an initial standard message, with the author supplying the remainder. For example, if the standard part of a message were "Press any key to ", the author could call the function thus:
pause("return to previous section");
the message that would be displayed would be:
Press any key to return to previous section
This would be a little more meaningful than the present standard message of "Press any key to continue".
Wildcharacters and wildcards are represented by ? and * respectively. There is no provision for these characters to represent themselves, and this is a limitation which should be rectified in future versions of PAL.
Some of the ideas that are presented here have similarities to ones which were considered (see 'A brainstorm session', chapter 3) for use in the PAL system. They have been included here for completeness.
In order to facilitate the transportation of the PAL package into non-English environments, the author error messages should be separated from the functions in which they are raised. There should be (as is described in the specification) an author error function which displays author error messages. These messages could be kept in a text file, so that it would be a simple matter to produce different versions for different languages.
There should be a way of saving the screen (and header and message displays) before the output of an author error, so that after the message has been shown, the previous screen is restored.
A syntax checker for PAL programs would be a useful addition to the system. It could be a separate program which is given the name of a text file (ie an author's code) as a command line parameter. If it were implemented, many of the Author Errors which are currently incorporated into functions would be redundant, so there would be a reduction in code size. In addition, it could be made to check for attempted branching to non-existent labels. Of course, this is done within the C environment, but not in a particularly user-friendly manner. Unused MATCHRES and ANSWER variables could be located.
In some cases an author might wish to change the definition of punctuation that is used with the author language functions. There should be provision for this with a suitable function which is external to the author language functions. The function could read a set of punctuation characters from the keyboard, and place them in a text file. When a PAL program is compiled this text file could be used to supply the definition of punctuation, and thereafter would not be needed. In this way, different programs could have different definitions, and there would be no necessity for external files after compilation has been completed.
In a similar manner, there could be provision for an author to re-define the wild card and wild character. This would be useful if a given subject area made use of these characters extensively.
Within the intelligent spelling assistant system, there could be a dictionary of commonly misspelt words (which might be dependent upon the CBT application context). This should probably be accessible to authorised users, so that it could be updated/modified as the need arose. Different languages should be catered for.
One difficulty that could arise with matching numbers is that a response might be written in words or figures. Although an author could of course specify which form is required, a system which could deal with both forms without raising an error would be a decided advantage. One solution would be to have a mapping of words to figures. This could be in a text file, so that any language that uses roman script could easily be catered for. With languages that have reasonably regular rules for the syntactic construction of written numbers a set of those rules could be incorporated into an intelligent system. An alternative would be to specify each word-figure relationship individually. This, however, would be quite limiting, and perhaps only numbers up to one hundred should then be considered.
Related to this is the possibility of dealing with ordinal and cardinal numbers (this might not apply in some languages).
The current version of the spelling assistant is quite primitive (and the implementation even more so!). As has been mentioned in the discussion of spelling assistant, ideally, for an advanced system, we want to separate domain specific rules (ie language and keyboard) from the inference engine. if this were done, then we would be able to alter the set of spelling rules that we use with relative ease, so that languages other than English could easily be catered for, and also, that within an English language environment, additional or refined language spelling rules could be incorporated into the assistant.
If any of the constraining match parameters are used (ie B, C, or P), then the spelling assistant is disabled. In some cases an author might not want this, so a set of over-ride rules incorporated into the system would be appropriate. With these an author would be able to state that, for example, the spelling assistant should be enabled, but that the case of an answer is important. If the spelling were correct, but the case wrong, a suitable message, or match result would be generated.
The introduction to this dissertation (section 1.4) stated that there were two major problems with currently available authoring languages, namely poor response analysis, and the limitations to the production of cheap mass-market CBT materials that is caused by the necessity of paying licence fees, or royalties. Solutions will now be discussed.
The production of a complete authoring package is a time-consuming process. A number of systems on the market already have many good facilities, and rather than trying to compete with these, I suggest that Dean Associates complement what is available. To this end, the development of a model/answer matching interface would be appropriate, since it is in the analysis of student's responses that most commercial authoring languages are deficient. Any authoring language which allows the use of external programs (and the passing of parameters to these) could be used in conjunction with the model/answer comparison functions. This is the solution to the first problem.
The second problem is not resolved in such a clear-cut manner, nevertheless, it is not insurmountable, and indeed, it may even lead to a commercial advantage. Since all the commercial packages that have been investigated are lacking in response analysis functions, it may be that there is a market for such functions, which Dean Associates could satisfy; marketing of suitable functions could offset the costs of royalties to other author language producers, and indeed might even lead to clear-cut profit. Of course, a more detailed feasibility study would be required before any definite conclusions could be reached.
The ideas considered in section 8.2, and the recommendations to the company (8.3.1), have been based on the assumption that what is required by authors is a pattern matcher for (usually) single word models. If, however, we break away from this assumption, then the use of intelligent systems which can deal with language at a deeper level should be examined. For practical purposes, this need not be a fully developed sentence parser, which can cope with all types of input; for the time being let that remain in the research laboratories. We need something that is commercially realisable. Section 3.3.8 dealt with the use of Augmented Transition Networks in restricted domain environments, and examples of working systems were given. Therefore, the investigation of the use of ATNs for sentence level model/answer comparison functions should be considered.
In spite of both careful investigation into the requirements of the company, and detailed design, problems arose during and after implementation of the functions.
On occasion, the company changed its requirements. This was not a major difficulty, as all the changes asked for were relatively easy to implement. However, this is a potentially dangerous area, which in a commercial setting could lead to acrimonious situations. To prevent this, the requirements should be discussed in great detail with company representatives, before the document is finally approved. Any requests for changes should be noted; if there is time, or for the purposes of goodwill, these might be implemented. Alternatively, future versions of the software could contain these modifications.
In this project the discussion of the requirements was minimal, and although the company project supervisor approved the Requirements Document, the formality of 'signing off' was not complied with.
A modular approach was taken to the design and implementation of the functions, and this worked well. It was a simple matter to modify/upgrade individual functions in isolation from the others. However, perhaps the ease with which this was done could be partially due to the nature of the task itself; after all, the requirements called for the development of a set of functions, which by their very nature are modular.
Some problems should have been spotted at an early stage, since they were, on reflection, so obvious. (For example, the problem of an author's not being able to use the function message, because at all times the system is displaying a message). This does not mean that detailed investigation and design is not useful, but rather, that there should be more of it. This could be a problem in a commercial environment, if an individual (or group) is working directly for a company, rather than as an external 'consultant'. Pressure to 'get some code written' should be avoided as far as possible. On the other hand, there is a case for rapid prototyping of selected functions/programs, or parts thereof. A suitable case in point is the screen layout, (and user responses), which is difficult to visualise on paper.
Careful scheduling is vital to the success of a project, if it is not to run way over financial and temporal budgets. Although in the case of this project, the time constraints were fairly strictly adhered to, this was only at the expense of not completing all the requirements. Commercially, this is not good practice. We should not attempt to do more than we can reasonably expect to do, and there should be a suitable margin of safety built into all our projections.
In comparison to many commercial projects, the development of PAL author functions was a minor exercise, and therefore, the conclusions drawn in this section cannot be automatically scaled up to large projects. This does not mean that they are invalid for large projects, merely that they should be used with care.
Preface | Contents | 1 Introduction | 2 Review | 3 Req. analysis | 4 Req. documents | 5 Specification | 6 Design | 7 Verification | 8 Discussion | 9 PAL manual | Appendix A | Appendix B | Appendix C | Glossary | References | Index