Instructors often ask for proof that zyBooks work. We'll state up front that we can't "prove" anything. But, there seems to be substantial and varied evidence that zyBooks help students learn over traditional textbooks/homework.
Studies
Various people, including us, have published research. Below is a sampling:
- Randomized controlled study: 250 students, all given a pre-quiz on a CS topic. Half then given digital access to best-selling traditional book's section on that topic, half given zyBook section (participation activities only). All given post-quiz. zyBook learners averaged 16% better. Even more interesting: The initially weakest students averaged 64% better, nearly matching the average. And, zyBook learners voluntarily spent twice as much time learning. See ASEE 2014 paper (received best paper award).
- Cross-semester analysis: By 2014, we had 100 universities using us. Four had replaced their textbook by a zyBook with these features:
- (1) Mature class that was run the same each year
- (2) Taught by the same instructor before/after
- (3) Taught in the same semester before/after (fall-to-fall, or spring-to-spring)
- (4) No other changes were made in the course.
- Those four courses were from U Michigan, UC Davis, and U Arizona (2 courses there), totaling 2,000 students. We analyzed student performance before/after, and found on average a 0.3 grade point improvement (on 4.0 scale), with 14% improvement on exams, and 7% on projects. In all four courses, the instructors attributed the improvement to students coming to class more prepared, due to zyBooks content being more engaging and effective plus students earning points for completing zyBooks activities. See ASEE 2015 paper (received best paper award).
- Dozens of studies provide similar results overall or on specific features. For example, McKinney also found a 0.3 grade point improvement when switching, plus improved pass rate from 78% to 91% (ASEE'20, to appear). Liberatore found strong correlation between effort spent doing zyBook activities and final course grade (ASEE'17). Edgcomb found students voluntarily did zyBook activities earnestly rather than rushing through to earn points (SIGCSE'17). Dozens more publications around zyBooks effectiveness exist. Search for "zyBooks" at scholar.google.com. Also see Frank Vahid's publications page.
Feedback from students and instructors
Student feedback suggests students recognize the learning benefits of zyBooks versus traditional textbooks. Before a course at UC Irvine decided to switch to zyBooks, they surveyed their ~400 students as to whether the required textbook contributed to their success in the course. Then they surveyed their ~400 students the next quarter using zyBooks. Before, students were all across the range of agree/disagree. After, nearly everyone agrees that zyBooks help. Oh, and BTW, while nearly 100% of students subscribed to the zyBook, 18% of students in the prior quarter's survey admitted they had NO access to the textbook (not even illegal pdf, which 30% of students admitted they had).
We survey students annually and they tell us zyBooks are better than textbooks, with 42% saying "dramatically" better.
And students commonly send us unsolicited emails to thank us. We have many hundreds of such comments.
- I've told all my professors and friends about how amazing the books I've used so far are. Keep up the good work, you're making a difference in this student's education :)
- I've found that I've excelled in the class and credit most of my success to the fact our professor taught through zyBooks ... I have tried other self taught platforms but none are as engaging or as challenging.
- I felt like it was working with me, rather than talking to me.
- It was a truly rewarding opportunity because I learned the entire language from your book with minimal instruction from my teacher.
- It is an understatement to say that I am in love with this book! I am learning concepts at an amazing rate!
- I have loved the books I have bought so far, and am recommending them to my friends and professors. Keep up the good work, I am very interested to see what other books you all make!
- I love using zyBooks. It's like codeacademy but a million times better and nothing at all like it. :)
- I genuinely cannot believe more of the CS department hasn't been using these books (and I'm sad I went through so many of my CS courses without this resource).
In fact, faculty have reported that students compliment zyBooks in their teacher evaluations, with comments like these:
- Zybooks and this teaching method-- learn at home and explained in class-- made this introductory Computer Science course a wonderful experience.
- Learning coding via Zybooks was also very enjoyable because the website itself was well-laid out and very clear.
- I loved the online Zybooks approach. It is definitely the best online learning platform of the ones I have used.
We have a similarly long list of positive comments from instructors who use zyBooks, but for conciseness we'll move on.
More on grades and student feedback
The company's co-founder is a professor at UC Riverside. Their CS1 course (serves about 1500 students per year) has been using zyBooks since 2013, introducing each new feature as they appear, including integrated homework problems, and then auto-graded programming assignments, and they have researched practices for using such features and adopted those found to be best practices. Their grade distribution has improved quite dramatically, with an exceptionally low DFW rate for a CS1 class, especially one taken by both majors and non-majors.
The anonymous end-of-term course evaluations have also steadily risen, from a 3.7/5.0 in 2010, ranking it in the 50th percentile, to a 4.7/5.0, ranking it in the 85th percentile -- again, an exceptionally good result for a CS1 class, which research shows often has a high fail rate and poor evaluations at schools across the country.
Retention
A key reason the company was founded was to improve retention of CS/STEM students in the first two years of college, since data showed about 50% of students left such majors in the first two years, and research showed a key reason was harsh grades and/or poor performance in early classes. Retention is very hard to study because there are so many confounding factors. But we get asked repeatedly for retention data. So, we looked at CS first-year retention at the co-founder's school (UCR) where they switched to zyBooks for their CS1 and CS2 courses -- a big part of a CS student's first-year experience. Upon switching to zyBooks in 2013, then integrating new zyBook features as they appeared (homework, program auto-grading), and adjusting teaching to use best practices around zyBooks, one sees that the first-year retention rate steadily rose since 2013 to about 90% and has been steady there ever since. And by the way, while females used to leave at nearly twice the rate as males, now female retention is the same as males -- in fact, it's a few percent higher.
We cannot claim the CS first-year retention increase is entirely due to zyBooks. Many other factors contribute, such as more stringent admission requirements, better student advising, etc. However, we compared to other majors in the same college, whose students would have had the same improvements in such other factors, and the CS improvement strongly outpaces those other majors in the college. Even then, we still can't claim the improved retention is due just to zyBooks, but the data does seem to suggest that strong use of zyBooks, along with really good teaching practices, can help improve retention.
Acknowledgments
And finally, we'll end this article by expressing appreciation. The above results could not have been obtained if not for much help we've received along the way.
In particular, the National Science Foundation strongly supported us via two US NSF SBIR grants that totaled over $2 million. The importance of this support, both financially and other support that such grants end up providing, cannot be overstated. We are DEEPLY grateful for this support. We also received an SBIR grant from the US Dept. of Education. And at UCR, the zyBooks co-founder received grants from the NSF and from Google to perform general CS education research around zyBooks. Plus, our seed and Series A investors also took a risk on us, enabled us to grow, and provided sage advice along the way. For all these, we are sincerely thankful.
We also wish to thank our instructors. To those who adopted us in 2013 and 2014 -- Each such early adoption was tremendously important in helping us grow, probably much more than you realize. Where we are today -- having served 900,000+ students at over 900 academic institutions -- is in part due to you taking a chance on us. Thank you, friends. And to those 2,000+ instructors that we serve today, we wish to thank you for supporting us and for being so wonderful about making suggestions in a professional positive manner. We always want to improve and we appreciate you supporting us, so that hopefully we can better serve you and help you serve those wonderful students of yours.