Dr. Badri Adhikari is an Associate Professor of Computer Science at the University of Missouri-St. Louis. His research interests include education, explainable machine learning, and health and bioinformatics. He has published dozens of papers in journals such as Education Sciences, Scientific Reports, Bioinformatics, Plos ONE, and BMC Genomics. He is the author of the widely used protein 3D modeling tool CONFOLD.
His research is funded by the University of Missouri-St. Louis (UMSL), NVIDIA, Google Cloud Credits, the U.S. National Science Foundation (NSF), and the National Aeronautics and Space Administration (NASA). At UMSL, he enjoys teaching Deep Learning, Artificial Intelligence, and Interpretable Machine Learning. His favorite course to teach is Data Visualization.
Tell me about your journey in computer science. What inspired you to enter this field? What are your main research interests? How do you integrate your research with teaching?
At sixteen, I fell in love at first sight with a Pentium II desktop computer. My first experience was playing a bike racing game called Road Rash. Coming from a world of black-and-white television, the bright screen, the sounds from dedicated speakers, and the endless keyboard buttons were awe-inspiring. Today, seeing people amazed by AI and its capabilities reminds me of my own childhood wonder with computers. While my love for computers has fluctuated over time, it has never truly faded.
My current research focuses on a subfield of AI called Explainable Artificial Intelligence (XAI). XAI aims to understand how and why AI works the way it does. I also teach a graduate-level course on this topic at UMSL. Other courses that I teach include AI, Deep Learning, and Data Visualization, and because all these courses are related to my research, integrating research into teaching is easy.
Can you explain the concept behind Process Feedback? What sparked the idea for this project?
Process Feedback is an educational tool designed to help students understand why they excel or struggle in their writing and coding tasks. Students can use it to write, code, or analyze the process behind their work.
The tool sheds light on students' writing or coding processes by transforming process data into meaningful summaries and interactive charts. This empowers students to reflect on their work and identify the reasons behind their successes or challenges. By providing insights into process data (breaks taken, typing fluency, revision time, etc.), Process Feedback encourages students to engage in deeper reflection and self-directed learning.
During or after writing activities, Process Feedback visually displays students' process details. Students can explore and learn from metrics like time spent on each paragraph, copy-paste events, and revision habits. Additionally, they can download a comprehensive PDF report containing all process data and charts, which can be shared with educators. This "process report" equips educators to offer personalized feedback that addresses both the outcomes (writing or code) and the underlying processes used to achieve them.
The launch of ChatGPT in November 2022 sparked a concern within me. As an educator who had dedicated years to crafting challenging and effective assignments, I worried about AI diminishing the value of fundamental cognitive and computational skills. I disagreed with simply "embracing" AI without considering its potential impact on student learning. After two months of wrestling with these concerns, I came up with the idea of Process Feedback. Regardless of whether students use AI, I reasoned, that if I could see their coding process, I could identify their learning gaps and provide targeted support. This concept extended to writing as well. The initial development involved several students and volunteers and took four months of intense work.
What are the main objectives of Process Feedback, and how do you aim to achieve them?
The primary objective of Process Feedback is to help students develop a thinking skill called metacognition. It is the ability to think about their own thinking. It's essentially the power to reflect on their work and identify areas of success and struggle.
Imagine a student who excels at creating first drafts but struggles with revision. Through self-reflection, they might realize they haven't thoroughly revised their writing. This reflection encourages them to critically analyze their process and identify what went well and where improvement is needed. This practice of looking back and analyzing one's thought process develops metacognition.
Process Feedback helps students improve their metacognitive skills by providing them with a platform to practice self-reflection in writing and coding tasks.
With AI becoming increasingly powerful, many cognitive tasks can be delegated to these systems. This begs the question: is it enough for students to simply learn "thinking" skills? Absolutely not. Students need to learn meta-thinking, which goes beyond basic thinking and involves reflecting on the thinking process itself.
Process Feedback helps students improve their metacognitive skills by providing them with a platform to practice self-reflection in writing and coding tasks. Visualizing their writing process fosters a deeper understanding of their approach. This unconsciously trains them to improve their workflow in the future. Furthermore, developing this habit of "looking back" becomes self-reinforcing. Once students start cultivating these higher-order thinking skills, they can apply them to various aspects of their lives and careers.
Can you describe the collaboration process with Missouri Online during the UX testing and feedback stages? How did our designer’s input influence the final design and functionality of Process Feedback?
As soon as I decided to build Process Feedback, I felt a sense of urgency to get it into users' hands. My background in AI and bioinformatics wasn't much help. With the core engine under development, the lack of a user-friendly interface felt like a roadblock. Luckily, at a meet-and-greet event, I met Missouri Online's design manager, Brad Mitchell. I expressed my need for help, and thankfully, he offered to assign a designer to the project. Relief washed over me as I left UMSL’s Lucas Hall that day.
Within a week, Rio was assigned to assist me. During our initial meeting, I explained my vision. She listened attentively, absorbing my ideas. Shortly after, she sent me the wireframes for all the application pages along with a style guide (a new concept for me at the time). These elements became the foundation for the first version's UX, launched in April 2023. While the design has evolved, Rio's contribution was essential. Without her expertise, the development process could have been significantly delayed.
Reflecting on Missouri Online's specific feedback, such as identifying user pain points and enhancing accessibility features, how have these improvements shaped the user experience on Process Feedback?
The Missouri Online UX feedback report was a revelation. Reading the report, I realized how much we'd missed. We hadn't paid enough attention to the finer details of the user interface. The report made it clear how little things can significantly impact a user's experience.
In retrospect, the true value of collaborating with Missouri Online wasn't just the deliverables – the wireframes, style guide, and report. It was the "Design 101 education" that I received. The design process and interactions with the team instilled in me the importance of user-centric design, maintaining a consistent layout, and strategically placing key features for optimal accessibility. I'm incredibly grateful to the Missouri Online team, particularly Brad and Rio, for showing me the way.
How does Process Feedback stand out from similar resources or platforms?
Process Feedback stands out for three key reasons. First, it's completely free and highly accessible. Unlike most educational tools with usage fees, Process Feedback requires no account creation and can be used immediately. Second, it prioritizes student privacy. The tool doesn't track every button click – a student's data remains in their browser until they choose to save it on our server. This allows students to self-record their process and save it selectively, unlike many tools that use Google Analytics or similar trackers for constant user activity monitoring. Process Feedback also ensures that students always see their data before sharing it with others.
Finally, Process Feedback boasts a wealth of features. It displays nearly all process summary data and visualizations that other tools often lack. For instance, we recently integrated AI features, transforming our coding platform into a world-class online compiler for learning to code – a capability typically found only in many paid tools.
Many teachers have shared that Process Feedback serves as a better alternative to plagiarism detection tools like Turnitin or GPTZero. While these tools focus on "detecting" plagiarism after the fact, Process Feedback has the potential to "prevent" it altogether by encouraging self-reflection and ownership of the learning process. Additionally, while plagiarism checkers may encourage originality, they don't directly contribute to analyzing and improving student learning as Process Feedback does.
What challenges have you faced while developing Process Feedback, and how have you addressed them?
Developing Process Feedback has been a whirlwind. Initially, technical hurdles dominated my focus. I tackled these by reaching out to experts, friends, and students for help, while simultaneously learning to become a full-stack web developer myself. Now, with a functional tool, other challenges have come to the forefront. Funding is a pressing concern. Currently, I rely on personal resources, but this isn't sustainable in the long term. Grant applications are underway, and I'm confident we'll secure funding for operational costs within a year or two. Generating media coverage is another hurdle that we have yet to overcome. However, I'm optimistic that this too will subsite organically over time. While daunting, these challenges fuel my motivation. Each morning, I'm eager to tackle them one at a time and keep Process Feedback moving forward.
For instructors looking to enhance student learning or promote academic integrity, Process Feedback can be a game-changer.
How can faculty get involved with Process Feedback now that UM System's IT compliance has approved the tool?
Yes, Process Feedback has been approved by UM System's IT compliance for use by all faculty and students across all four UM campuses. Faculty can get involved in two key ways: integrating Process Feedback into their teaching or using it to research student learning. For instructors looking to enhance student learning or promote academic integrity, Process Feedback can be a game-changer. By incorporating it into their courses, faculty can help students develop their writing or coding skills in a new way. Process Feedback can be particularly beneficial for courses that focus on improving these skills. Alternatively, faculty can also use Process Feedback to demonstrate their own writing or coding approach to students, aiding a deeper understanding of the process.
What has been the impact of Process Feedback since its launch? Are there any success stories or particular achievements you’d like to share?
Launched in April 2023, Process Feedback is making steady progress. Around 600 students – 200 in Nepal and 400 in the U.S. – from various schools and universities are actively using the tool. We've facilitated over 20,000 student assignments so far. Additionally, I'm presenting Process Feedback at the Canada International Conference on Education in Toronto and MIT's AI & Education Summit in Cambridge this July. Several conference papers are also under review.
One impactful story involves engineering students at Advanced Engineering College in Nepal. Traditionally, to prevent plagiarism, students submitted handwritten coding assignments. This, however, hindered their learning, as code is meant to run on computers, not paper. Recognizing the value of Process Feedback, the college has transitioned entirely to the platform, eliminating hand-written reports. As a former student of Advanced Engineering College, I'm proud to contribute to their progress.
How do you measure the success and effectiveness of Process Feedback?
Process Feedback's success hinges on its impact on student learning. We aim to empower students and actively engage them in their own learning journey. Therefore, we measure success not only by the number of users but also by how effectively the tool helps students learn and develop metacognition skills.
Recently, we integrated an "explanatory AI" feature to help students understand code and error messages. We surveyed 100 students each from Nepal and the U.S. who used Process Feedback to learn coding. While all the students found it helpful, those from Nepal, who were unacquainted with such capabilities, were particularly grateful. This is a small win for us. A full research paper describing this work is currently under review for an international conference.
However, to scientifically assess the tool's effectiveness, a randomized control study is needed to quantify the benefits of using Process Feedback. This is considered the gold standard in research. We plan to apply for funding to pursue such a randomized control trial in the future.
I firmly believe that true understanding stems from self-reflection.
Can you share any feedback from users of Process Feedback that has particularly resonated with you?
The impact of Process Feedback truly resonated with me through a story from a computer science professor at Saint Louis University. He immediately integrated the tool after I introduced it, and within a week, he shared his students' enthusiastic response. This piqued my curiosity to learn more about his unique implementation.
During our conversation, he described his approach. He began by showcasing his own coding process live in class, followed by a review where he discussed strengths and areas for improvement. He then encouraged students to do the same during their bi-weekly presentations. Previously, students had difficulty finding presentation content, but Process Feedback's visuals and data provided them with ample material.
I firmly believe that true understanding stems from self-reflection. Encouraging students to reflect and present their findings is a powerful approach to deep engagement. This professor's approach perfectly aligns with my philosophy on how learning occurs.
What are the future plans for Process Feedback? Are there any upcoming features or tools that users can look forward to?
Process Feedback addresses a key issue in education: incorporating AI responsibly in the classroom. Many educators discuss using AI with "guardrails," but few tools provide a way for students to use AI in such a way. We're transforming Process Feedback into such a platform.
Our online compiler already integrates safe AI features. Students can use AI to understand cryptic error messages from the compiler in plain English or to explain any piece of code clearly. We're developing a similar feature for our online writing editor.
What excites us more are two innovative features in development: Process Feedback for Google Docs and Google Colab. This summer, we plan to release a Google Chrome extension, allowing students to write in Google Docs while gaining insights into their writing process through Process Feedback. Similarly, computer science students learning Python will be able to leverage Process Feedback with Google Colab.
How do you envision the growth of Process Feedback in the next five years?
In five years, I see Process Feedback becoming the standard for thousands of courses worldwide, across schools, colleges, and universities. With its focus on self-reflection and independent learning, Process Feedback revolutionizes introductory writing and computer programming courses, empowering tens of thousands of students to write and code better. I envision Process Feedback's adoption growing exponentially at around five years from now.
Now that AI can code computer programs, some leaders in computer education suggest that we might stop teaching computer programming fundamentals the way we do today. I plan to actively stay up-to-date on new and emerging educational trends and adapt Process Feedback to such potential changes in practices, continuing its core mission of nurturing student reflection and self-learning through process analysis.
Are there collaboration opportunities or contributions you would like to invite from the UM System community?
Process Feedback bridges the gap between various research areas, including AI, writing, coding, data visualization, communication, education, and academic integrity. I'm actively seeking collaborators from any discipline to work on research projects or write grant proposals. With a functional tool showing great promise, we have a strong foundation for groundbreaking work together.
Final thoughts
The success of Process Feedback wouldn't have been possible without the incredible support of many people. Students, friends, colleagues, even acquaintances and family – all have contributed along the way. My deepest gratitude goes to them. A belief in the power of the idea and the potential of the tool fuels my relentless pursuit of improvement. I encourage anyone reading this to share their thoughts–your feedback is valuable.
Request Support
Our design services are available to all faculty from any of the four UM System campuses. So why wait? Let's get started today!
Author
Brad Mitchell, Ed.D.
Brad Mitchell is a Design Manager at Missouri Online and serves as an adjunct instructor in the Missouri School of Journalism. His research interests include media literacy education, educational media pedagogy, and higher education leadership.
