Any software engineer in industry can tell you the importance of software testing in the software development lifecycle. The quality of the software is ultimately the deciding factor on whether the software is ready to ship. Yet, very little time of a computer science student’s studies involve software testing. On any one of my programming assignments, there is at least one or more cases of a student’s submission either crashing shortly after startup or producing incorrect results on valid inputs. When I circle back with these students on how this happens, the common response is that they did not think to test the particular scenario. A group of computer science professors at the University of Texas have found a possible reasons for this result.
Software testing is a subject that can be difficult to teach, perhaps because it relies heavily on experiential learning; at the same time, because it is an activity that most students do not enjoy, students tend to expend minimal effort on testing.
(Smith et al., 2012)
Software professionals are acutely aware of the importance of software testing. In the days of the waterfall software methodology, software testing was a distinct, end stage of the development process that was often performed by a separate set of software engineers. Bill Gates made this observation about testing at Microsoft:
Microsoft in terms of this quality stuff — we have as many testers as we have developers. And testers spend all their time testing, and developers spend half their time testing. We’re more of a testing, a quality software organization than we’re a software organization.
(Foley & Murphy, 2002)
Today, the software industry has moved to an agile software methodology that eliminates the distinct testing stage found in the waterfall model. Developers are fully responsible for developing, running, and passing all tests on their code before completing a sprint. This model puts much more pressure on software engineers to produce quality test cases – a task that is currently missing in most computer science curricula.
So, how do we get computer science students to think more about software testing? The start of an answer can be found by applying computational thinking defined by the International Society for Technology in Education (ISTE) standards for students (ISTE, n.d.). Educators define computational thinking as the process of applying computer science concepts like decomposition, pattern matching, and abstraction to learning disciplines other than computer science. However, these computational thinking concepts are very relevant to the process of software testing. Computer science students can apply automation to the process of software testing and algorithmic thinking to develop a sequence of steps to create automated test cases.
Professional software engineers use a test harness or framework to manage test cases. These test frameworks can be complicated to setup and manage. Fortunately, there are tools that are much more approachable for computer science students. One such tool is Jasmine, an open source test framework for building JavaScript test cases. There are some terrific Jasmine tutorials that can greatly lower the learning curve for computer science students, and Jasmine is being used with modern JavaScript libraries like React and Angular.
Just making a test framework available to computer students is not sufficient to get students to write useful test cases. While test frameworks simplify the task of writing and managing test cases, the students still need motivation to write decent test cases. I researched how other instructors approached this problem – and what techniques were used to motivate students to write better test cases. I wanted my computer science students to foster the same work skills of collaboration, problem solving, and project management that is often found in many robotics classes.
Robotics is also a highly effective way to foster essential work skills like collaboration, problem solving and project management. It does all this while keeping kids so motivated and engaged that getting them to stop working and move on to the rest of the school day can be a challenge — a good problem to have!
(Gura, 2013)
A great article that shows promising results in this area uses peer review or peer testing, a process in which students try to break code written by their peers (Smith et al., 2012). The study was done by instructors in the Fall quarter of 2011 at the University of Texas at Austin for CS314H, an honors data structures course in computer science. The goals of their approach matched my goals.
Our solution has three main goals. First, we want to make testing fun and competitive so that students will put effort into testing. Second, we want students to learn from each other, so that they can see how others approach the same problem, perhaps with a greater degree of creativity than they have. And third, we wish to illustrate the tangible benefits of good software testing by uncovering latent bugs in their code.
(Smith et al., 2012)
The instructors found that a majority of the CS314H students enjoyed the peer testing exercise and found it worthwhile. Peer testing did involve more work for students – yet many of the CS314H students expressed interest in doing more peer testing. This is a wonderful result, and something I hope to replicate in my computer science courses!
References
- Barr, D., Harrison, J., & Conery, L. (2011). Computational thinking: A digital age skill for everyone. Learning & Leading with Technology, 38(6), 20-23.
- Foley, J., Murphy, C. (2002). Q&A: Bill Gates on trustworthy computing, Information Week.
- Gura, M. (2013). Student robotics and the K-12 curriculum. Edutopia. Retrieved from https://www.edutopia.org/blog/student-robotics-k-12-curriculum-mark-gura
- Grover, S., & Pea, R. (2013). Computational thinking in K–12: A review of the state of the field. Educational Researcher, 42(1), 38–43.
- ISTE Standards for STUDENTS. (n.d.). Retrieved January 15, 2019, from https://www.iste.org/standards/for-students
- Smith, J., Tessler, J., Kramer, E., & Lin C. (2012). Using Peer Review to Teach Software Testing. ICER’12, p. 1-6.
Nat , I always enjoy reading about how you apply the ISTE standards in your classroom to better prepare your students for the professional field they will embark upon once they graduate. Your students are lucky to have such a dedicated professor who is keeping up with how jobs in the tech world are evolving and they can evolve with these changes throughout their careers. You have come up with a solid way to incorporate Computational Thinking as well as introduce them to a new tool while including the very important WHY and HOW component!
Nat,
From your paper, I can see the software testing is a process relevant to computational thinking concepts. I like the project of peer-testing you mentioned. It makes software testing fun and motivated. They can learn from each other’s different approaches anchored the same problem which can foster their algorithmic thinking to develop a sequence of steps to create and test automated solutions. Thanks for sharing.
Nat, this post is so interesting – sw developers are expected to test their own code now – no more testing groups? It must be so difficult when new CS graduates start working and they haven’t been exposed to this in their programs.
The idea of peer testing is brilliant – immediate feedback with a dash of competition. By doing this you will be giving your students a big advantage when they start to look for jobs, as well as once they’ve landed them.
Nat,
I appreciate your intentional practice of computations thinking skills within job ready classrooms. There is value in taking the academic practice and applying it to workforce readiness. I enjoy reading about how you interpret the learning with your students in mind. You are providing valuable resources and opportunities for meaningful practice in your classrooms.
Nat,
Your posts are always so informative. You have given the history of trends in software designing and made an argument of why it is important for students to test their code. I was able to make a connection between this problem and my students not wanting to go back and make edits to their papers. In both cases, I believe it when you quoted Smith, to say this is difficult to teach and students do not enjoy the process. I like how you included the vision of software testing at UT Austin, finding a way to make it fun and competitive while learning from each other is a great way to encourage this shift.