Researchers Chin-Ying Liew, Kien-Kheng Foo, Elinda Ai-Lim Lee and Kelvin Tee-Hiong Goh studied the types of systems-thinking skills used by 233 students in the fields of Social Sciences and Humanities to solve non-routine problems. They used five systems-thinking skills in the framework - Dynamic Thinking Skill, System-As-Cause Thinking Skill, Forest Thinking Skill, Operational Skill and Closed-loop Thinking Skill.
A scoring rubric based on the systems-thinking framework was developed by the researchers because there was no ‘prescribed’ marking scheme for measuring systems-thinking skills available. An analytic rubric was used as the categories of skills that were to be graded. Each systems-thinking skill was subdivided into sub-skills. The checklists or sub-skills of the five essential systems-thinking skills were then carefully worded for all the four performance tasks. Hence, for each correct sub-skill, a certain point ranging from 1 to 5 marks was allocated.
Data were collected through two paper-and-pencil test sessions. A second session was conducted for those who could not make it for the first session. The respondents were gathered in a lecture theatre and were given 1 hour 20 minutes to answer all the questions.
Discussion and conclusion
The t-test result indicates that there is no significant difference between the scores for male and female (p > .05). The test was adequate to detect for significant differences. The findings indicate that the general population performed poorly (23.5%) for systems thinking in the four performance tasks. In a similar study, Ossimitz (2002) reported that the average performance on each of his tasks was below 45%. His sample consisted of Masters and First Degree undergraduates. Contrary to popular belief that Masters and First Degree undergraduates can outperform Diploma students, his sample performed only slightly better than the Diploma students in this study.
One important aspect here is that this study was actually an attempt to quantify their informal systems-thinking skills as none of the respondents had received formal systems thinking education. One thing for certain was that the instrument succeeded in capturing and quantifying these skills for comparative purposes. The instrument that was used here shared much similarity to that of Ossimitz (2002).
The findings revealed that gender was not a factor that differentiated the performance of the students in systems thinking. The study by Ossimitz (2002) reflected that the performance of the males in his tasks was consistently higher than that of the females though he accentuated explicitly the inappropriateness to, thus conclude that female subjects were generally inferior to their male counterparts. On the other hand, the study by Sweeney and Sterman (2000) disclosed that there was some marginal effect with males performing slightly better than females on all their performance tasks though the effect was reiterated as only marginally significant. Therefore, further research into gender difference should be carried out to ascertain more conclusive findings.
When the programme of study was used to compare the mean score of system-thinking skills, as in Table 3 and Table 4, it indicates that there are significant difference(s) between the mean scores. Following that, from Table 5, the result reaffirms that the mean score for Science and Technology was significantly different from the mean score of the Business and Management respondents. This result shows that the performance of the Science and Technology respondents is significantly better than students from Business and Management and Social Sciences and Humanities. The possible reason for this difference could be that the Science and Technology students received exposure to the basic concepts of behavior over time graphs, feedback, ordinary differential equations, and identifying units of measure, basic understanding of probability, logic and algebra and other systems-related mathematical concepts which are also the underlying concepts of systems thinking. This finding is very similar to that of Sweeney and Sterman (2000) where those with technical backgrounds did better than those in the Social Sciences and Humanities in one of their tasks.
When CGPA was used to make group comparisons, the result yielded a significant difference between those with a CGPA of 2.00-2.50 and those with a CGPA of 3.50- 4.00. Similarly, the difference was also significant for those with a CGPA of 2.50-3.49 and a CGPA of 3.50-4.00. However, the test showed no significant difference between the mean score of those with a CGPA of 2.00-2.49 and a CGPA of 2.50-3.39. Therefore, the finding indicates that CGPA has some influence on systems-thinking performance. The students with a CGPA of 3.50 – 4.00 outperformed those with a CGPA of 2.00-2.50 and also those with a CGPA of 2.50-3.49.
In conclusion, the results illustrate that among the three demographic characteristics, only programme of study and academic performance showed some influence on systems-thinking performance, while gender was not a factor in determining the difference of systems-thinking scores.
Limitations and recommendations
Though effort was made throughout the period of survey to ensure the best representative study, limitations were still found. It was clearly noticed that it was inadequate to accurately capture the qualitative nature of higher-order thinking process when a quantitative design was employed. Other limitations arose from the aspect of small sample size for Social Sciences and Humanities, the length and depth of performance tasks constructed, the scoring rubrics designed and the lack of incentives provided to boost the attention span and time of respondents. Lastly, the low scores reported in this study may not be due solely to the respondents’ lack of knowledge. As commented by Sweeney and Sterman (2000), “Perhaps people understand stocks, flows, delays, and feedback well and can use them in everyday tasks, but do poorly here because of the unfamiliar and unrealistic presentation of the problems”, the poor performance may be caused by the systems-thinking performance tasks given to the respondents were not put in ‘familiar and realistic situations’.
There remain still many outstanding issues that have yet to be resolved which can be recommended for further survey such as: does mastering more systems thinking implies better performance in the non-routine problems? Is good training in mathematics or sciences sufficient to understand the basic concepts of systems thinking like stock and flow, time delays and feedback to name a few? What other demographic variables influence the acquisition of systems thinking concepts? These are but a few of the issues that merit further research.
Contacts:
