Problem Solving in Community Practice

7. Problem Solving in community Practice

7.3.3 Extended Study on a Casual Basis - University of Brighton

The second year Pharmacy students at Brighton studied this package in a similar way to the first year group at the same site, who worked with the Guinea Pig Ileum package. Students were given four weeks in which to study the package in their own time. Pre-tests were carried out for the whole year group at the same time and place, immediately after a lecture. Students were then told where the package was to be located. No other help or instructions were given. Students were told that after four weeks there would be a similar post test and questionnaire. No observation of students took place for the reasons outlined in the previous study involving students working in their own time (section 6.3.4). Post tests were again conducted immediately following a lecture at which the entire cohort should have been present. Apart from this, the methods used followed the standard set of evaluation measures used in the other studies. One problem arose which affected the methods used. A discussion group was conducted after the post test but due to a malfunction, the session was not recorded on audio tape. Brief notes of the key points were hand-written, after this error was discovered but quotations can only be made from the questionnaire responses for this study.

There were again only a small number of valid cases, 33 out of a possible 85. A significant number of students were absent for pre-test, post test or both and a large number of students again did not do the package and their scripts have been excluded. Similar hypotheses to those suggested when this method was discussed in the section on the Guinea Pig Ileum package are likely, namely that there was some absence through illness, the nearness of exams may have kept some students away and some students probably stayed away deliberately. It is also possible that some students had worked on the package but were not present at the post test, because they were missing the lecture beforehand for other reasons. These second year students had already done their dispensing course and were therefore in a similar position to the students at Cardiff, in that this was to be largely a revision exercise. However, dispensing exams were due to begin shortly after the four week period and this helped to provide additional motivation for the students, although later in this section, several, other factors which played a role in the learning process, will be outlined.

Levels of computer use and confidence again showed that students are well prepared for working with CAL packages. 100% had used a computer and mouse previously. 54% used a computer "at least once a week". 94% felt at least "Fairly comfortable" about working with a computer . All students worked alone, only four (12%) of whom asked for any help and all help requests were addressed to friends.

FIGURE 16: CONFIDENCE LEVELS BEFORE AND AFTER TEACHING EPISODE (n=33)

click image to see larger version
Levels of confidence before studying the package are consistent with a group who had some prior knowledge in this area with 88% stating that they felt "Confident" or "Fairly confident" in the subject. After the four week study period, 62.5% declared that their confidence had increased, whilst only 37.5% said that they felt about the same. As with the students at Cardiff, this marked increase in confidence is surprising in a group with high levels of prior confidence. Table 20 below also shows that this confidence is not reflected in the results of pre and post tests. The paired t test reveals that the difference between the scores is significant, but in reverse, with pre-test scores significantly higher than post test ones.

TABLE 20: PRE AND POST TEST SCORES AT BRIGHTON (N=33)

UNIVERSITY PRESCORE Range 0 - 18 POSTSCORE Range 0 - 18 t Value P Value S.E.M

Brighton 11.8108 10.5405 2.75 .009 .461

If we examine the sections that the students worked through, the same picture as that of the previous study emerges, of students working sequentially through the material and missing out the last section. 97% worked through Prescriptions, 94% worked through Labels, 30% worked through Patient Information Leaflets. This, together with the data from the previous study at Cardiff gives a clear impression that students did not regard the third section as important or relevant to their needs. The Patient Information Leaflets section does not follow on directly from the previous two sections and is a slightly separate subject. In fact, the courseware developer responsible for this package, remarked that it was added later on. Clearly students have recognised this and not felt as motivated to study it, as the following comment suggests:

"I didn't have enough time to do patient information leaflets as I thought the scripts and labels were more important" - Questionnaire response

This is likely to have affected their scores, as the tests were based on all three sections of the package. Re-entering and analysing the scores section by section would help to determine this but this is not possible within the current scope of the project.

Given that this package was used as a revision aid by these students, it is appropriate to examine the results of those who began with a lower score to see if the learning outcomes for such students were any different to the group as a whole. Table 21 below sets out the results for those with a pre-score of 10 (55.5%) or less.

TABLE 21: PRE AND POST TEST SCORES WHEN PRESCORE <=10 (N=9)

UNIVERSITY PRESCORE Range 0 - 18 POSTSCORE Range 0 - 18 t Value P Value S.E.M

Brighton 9.1111 9.4444 -.30 .771 1.106

This table shows that students with scores at the lower end of the group have improved more than the group as a whole, although the improvement shown is still very small. Paired t tests on those scoring nine or less show a significant increase in scores on the post test but represent just three students which is probably too small a group upon which to make any judgements. A larger sample for this study would have made it possible to assess such trends more clearly.

Two further issues should be raised concerning the results on learning outcomes. Firstly, as mentioned in the section on Limitations (4.3), the pre and post tests can only be seen as a broad indicator of learning gains. The tests were deliberately designed to be short, with just one question on each objective to ensure the goodwill of students in undertaking the tests and to facilitate the necessary time-tabling for the studies. However, this does make them a fairly crude assessment tool. A more finely graded test may have been more beneficial in assessing students such as these at Brighton and those at Cardiff also, who already had significant pre-knowledge in this area.

Secondly, it must be remembered that the benefits and outcomes for the students may not be directly measurable, especially by pre and post tests, and that the increased levels of confidence and favourable comments from students do indicate that the package has been beneficial to this group. The following quotations support this:

"I found the package helpful and good for consolidating knowledge on prescriptions, especially before dispensing exams" - Questionnaire response

" It was a good package in the sense it gave an overview of what we need to know. I think it is very useful for 1^st year students to get the basic information on prescriptions, labels and PILs. It was useful to me, as a second year to test my law knowledge" - Questionnaire response

FIGURE 17: TIME SPENT WORKING WITH PACKAGE

click image to see larger version

Figure 17 shows the time spent working on the package and a varied pattern emerges, similar to the pattern of time spent on the Guinea Pig Ileum package by first year students at Brighton, again studying in their own time (see figure 12). However, in contrast to that study, these students have spent more time on the package, with 52% working for one to two hours on the package. This implies that those students who did work on the package were more motivated than the first year group studying the Guinea Pig Ileum. The following responses by students show the variation in time spent and how some students spent a number of sessions on it:

"~4hrs. Divided time about 40 min. at a time." - Questionnaire response

"Two 1 hour sessions - one hour on labelling, one on prescriptions - Quizzes at the end. Quiz was quite useful" - Questionnaire response

"I spent 3hrs doing the package and it is very good but I still only have the same amount of confidence because this area is more practical work" - Questionnaire response

One further issue that must be raised in connection with the methods used concerns other teaching or study material the students may have been exposed to during the four week study period. One limitation of this study is the fact that students were not asked to monitor or declare what other learning materials or interventions they had been exposed to. With hindsight, it would have been useful to gain some impressions of the other influences coming to bear on the study and this may have shed some light on the poor results of the pre and post tests compared to the increase in confidence recorded by those students who had worked on the package. However, it is still reasonable to assume that the students were studying other material, especially as the dispensing exams were approaching, and it may be that studying the same subject in a number of ways has proved counter-productive. There was evidence of this during the discussion group, when one student pointed out that the nearness of the exams meant that this was only one of a number of things they had been doing during this period, although no negative association was attributed to this.

In conclusion, it appears that the four week study period for students to work with this package has shown mixed results. Once again a large number of students did not study the package, despite the relevance of this to forthcoming dispensing exams. The majority of students who did study the package declared that their confidence had increased but the learning outcomes, evidenced by the pre and post test results show the opposite of this, with a significant decrease in scores. Learning outcomes for those who were weaker in this subject and began with a lower pre-score (<= 10) show that the package has been of more benefit to them. Failure to study the third section of the package may be partly to blame as only 30% worked on this section and yet the pre and post tests were based on the whole package. Students that did study the package, spent a reasonable amount of time on it and in some cases more than one session. Furthermore comments during the discussion and on questionnaires were favourable. Nevertheless the methods of using this package have had an effect on student learning and a more structured teaching scenario is more likely to ensure students achieve both their own and the tutor's objectives.