Current Projects

Dynamics of undergraduate student generic problem-solving skills captured by a campus-wide study

Andis Klegeris, Stephanie Barclay McKeown, Heather Hurren, Lindsay Spielman, Maegan Stuart, Manpreet Bahniwal

One of the most highly valued instrumental competencies expected of university graduates independent of their area of study is the ability to effectively problem solve.  Evaluation of the development of problem-solving skills is hindered by a shortage of widely available tools that could be used to monitor student progress, and by lack of well-defined instructional strategies designed to facilitate development of these skills.  Our current research project is aimed at addressing these problems.  We have developed an evaluation tool, which we applied to study the dynamics of problem-solving skills.  We tested first- and upper-year students from 26 different courses (total enrollment of 2,229 students).  Overall, improvement of problem-solving skills was detected for the first-year students over their first term of study.  There were no significant differences between the problem-solving skills of first- and upper-year students, and no improvement was detected for the problem-solving skills of upper-year students over one term of their studies.  Only three courses were effective at facilitating problem-solving skill development.  If these observations are supported by studies in other universities, it could be concluded that most of the standard lecture approaches are not effective at improving student problem-solving skills, and that universities and individual instructors must take active steps to advance this critical skill set in university students.  We are seeking collaborations to collect similar data in other universities; the generic problem-solving tests that we used are freely available to anybody interested in testing these skills of students in their classrooms or universities.

Use of diverse instructional and assessment techniques reveals distinct student skill sets

Andis Klegeris, Manpreet Bahniwal and Heather Hurren

Assessments conducted with increased frequency and variety within a course are believed to better reflect student achievement. Here we confirm that use of different instructional techniques allows students to develop diverse skill sets. Diversification of assessment methods is important for evaluating these skills. We performed a correlation analysis of marks in a pharmacology (BIOC 308) course which employed diverse instructional and assessment techniques (see table below).

Several statistically significant positive correlations as well as notable lack of correlations between sets of marks were revealed. These observations may indicate that different assessments measure student proficiency in different skill sets. Three categories of skills were identified: 1) study skills/work ethic; 2) group and social skills; and 3) problem solving abilities which correspond well with the categories of skills described in Tuning Methodology (see diagram below). Our data suggest that a detailed portfolio of marks documenting student’s achievements in an array of competencies could be a superior method of evaluating and communicating student performance than a single final course mark.

 

Positive effect of problem-based learning in a large classroom setting on student problem-solving skills.

Andis Klegeris, Manpreet Bahniwal and Heather Hurren

We have developed and tested two comparable sets of questions based on OECD-PISA tool that is not discipline specific and therefore can be used to assess problem-solving abilities of undergraduate students from different disciplines and backgrounds. The similarity of these two tests allows crossover design where students are offered one test before being exposed to PBL exercises and the other test after the exposure. We have used these tests on more than 200 students from four different disciplines. Our preliminary data show that PBL significantly improves student problem-solving abilities, while no trend towards the improvement is observed in classes using didactic lectures only. We need to perform additional studies to rule out possibility that the effect observed is due to different instructors delivering different courses and that it is PBL and not other components of the course that is causing the effect observed. This study is ongoing.

Please contact us if you have any comments or if you would like to participate in our studies.

"Daring ideas are like chessmen moved forward; they may be beaten, but they may start a winning game."
- Johann Wolfgang von Goethe
"An expert is a man who has made all the mistakes which can be made, in a narrow field."
- Niels Bohr