Acta Paedagogica Vilnensia ISSN 1392-5016 eISSN 1648-665X
2026, vol. 56, pp. 270–294 DOI: https://doi.org/10.15388/ActPaed.2026.56.15
Mykhailo Luchkevych
Department of Information Systems and Networks
Lviv Polytechnic National University
Lviv, Ukraine
luchkevychmm@gmail.com
https://orcid.org/0000-0002-2196-252X
https://ror.org/0542q3127
Viktoriia Luchkevych
Department of French and Spanish Philology
Ivan Franko National University of Lviv
Lviv, Ukraine
luch.vik11@gmail.com
https://orcid.org/0000-0002-3363-7791
https://ror.org/01s7y5e82
Abstract. This study examines the impact of artificial intelligence on the learning process in the field of DevOps, in particular, regarding its role in developing critical thinking, independent analysis and the ability to solve complex technical problems. The findings suggest that AI can improve the learning efficiency for students with medium and low levels of preparation, while reducing the motivation for independent thinking among high-achieving students. The use of AI in DevOps education accelerates task completion, but may lead to a reliance on automated solutions, which negatively affects the assimilation of complex concepts and long-term cognitive development.
Keywords: artificial intelligence, DevOps education, critical thinking, independent learning, automation of the learning process.
Santrauka. Šiame straipsnyje nagrinėjamas dirbtinio intelekto (DI) poveikis ugdymo procesui taikant „DevOps“ (programinės įrangos kūrimo ir eksploatavimo (operacijų) metodiką), ypač jo vaidmuo ugdant studentų kritinį mąstymą, skatinant savarankiškumą ir stiprinant jų gebėjimus spręsti sudėtingas technines problemas. Šiuolaikiniame ugdyme DI aktyviai naudojamas mokymosi procesams automatizuoti, turiniui personalizuoti ir kontekstui pritaikytoms rekomendacijoms rengti. Tačiau kyla klausimas, kokį DI naudojimas turi poveikį studentams, besispecializuojantiems „DevOps“ srityje, kur itin svarbu analitiniai įgūdžiai, gebėjimas priimti netradicinius sprendimus ir savarankiškumas.
Tyrime naudota mišri metodologija – mokslinės literatūros apžvalga, studentų apklausos ir eksperimentinis DI naudojančių ir nenaudojančių grupių mokymosi rezultatų palyginimas. Iš viso buvo apklausta 120 reguliariai su automatizuotomis sistemomis dirbančių inžinerijos studentų, o eksperimentinis testas buvo atliktas su dviem grupėmis – viena grupė užduotis atliko su DI pagalba, o kita be jos.
Tyrimas atskleidė, kad dirbtinis intelektas (DI) turi dvejopą poveikį mokymosi procesui. Viena vertus, DI gerokai padidina darbo efektyvumą, leidžia greičiau įsisavinti žinias ir automatizuoja rutinines užduotis. Tai ypač naudinga vidutinių ir žemų akademinių rezultatų pasiekusiems studentams, nes DI padeda jiems užpildyti žinių spragas. Kita vertus, dėl lengvos prieigos prie iškart sugeneruojamų sprendimų gerai besimokantys studentai gali prarasti motyvaciją savarankiškai atlikti analizes.
Eksperimento rezultatai parodė, kad dirbtinį intelektą naudojusi grupė užduotis atliko greičiau, tačiau 65 % dalyvių nesugebėjo paaiškinti, kaip jų sprendimai veikia. Tuo tarpu be dirbtinio intelekto dirbantys kontrolinės grupės studentai pasižymėjo aukštesniu savarankiško mąstymo lygiu ir gilesniu medžiagos supratimu – abu šie veiksniai yra kritiškai svarbūs veiksmingam „DevOps“ inžinierių rengimui. Ilgalaikiai rezultatai taip pat parodė, kad be DI mokęsi studentai geriau perprato žinias ir sugebėjo jas pritaikyti naujose situacijose, o tiems, kurie rėmėsi automatizuotais patarimais, dažniau reikėjo papildomų paaiškinimų.
Išvadose pabrėžiama, kad, norint veiksmingai naudoti DI „DevOps“ mokymuose, taikymą reikia adaptuoti. Užuot būtų tiesiog teikiami automatizuoti atsakymai, reikėtų taikyti kritinį mąstymą skatinančius mokymo metodus, pavyzdžiui, atviras projektines užduotis, situacines atvejų studijas ir grupių diskusijas. Be to, patartina įdiegti pagrįsto (angl. justifiable) DI naudojimo metodiką, pagal kurią, prieš gaudami DI sukurtas užuominas, studentai turėtų savarankiškai analizuoti problemas ir pateikti savo sprendimus.
Galima teigti, kad dirbtinis intelektas gali būti veiksminga priemonė rengiant „DevOps“ specialistus, jei jo naudojimas derinamas su savarankiškumą ir analitinį mąstymą skatinančiomis metodikomis. Ateities tyrimai turėtų būti sutelkti į ilgalaikį dirbtinio intelekto poveikį mokymosi procesui ir optimalių strategijų, skirtų jam integruoti į „DevOps“ praktikas, kūrimą.
Pagrindiniai žodžiai: dirbtinis intelektas, „DevOps“ mokymai, kritinis mąstymas, savarankiškas mokymasis, mokymosi proceso automatizavimas
__________
Received: 16/10/2025. Accepted: 17/03/2026
Copyright © Mykhailo Luchkevych, Viktoriia Luchkevych, 2026. Published by Vilnius University Press. This is an Open Access article distributed under the terms of the Creative Commons Attribution Licence (CC BY), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Artificial Intelligence (AI) is being actively integrated into various fields of human activity, including education. Modern technologies enable the use of automated learning systems, adaptive tests, intelligent tutors and algorithms which personalise the learning process. All of these contribute to superior understanding of material, an increased learning efficiency, and improved accessibility to knowledge.
The benefits of using AI in education are obvious: faster information processing, access to virtually unlimited resources, instant feedback, and personalised learning experiences. However, the question arises as to how such innovations affect the training of professionals in the field of DevOps – a field that requires a high level of independence, critical thinking and problem-solving skills.
DevOps professionals work in a dynamic environment where they have to adapt quickly to changes, understand complex technological processes, and automate IT infrastructure. as Along with the development and possession of technical skills, they need to be able to analyse problems in depth, anticipate potential outcomes, and make the most appropriate decisions. However, with the widespread adoption of AI, the learning process is changing: technologies can take over part of the cognitive workload, provide ready-made solutions and, as a result, influence the depth of material understanding and the development of independence.
A particular concern is the impact of AI on the motivation of students studying in the DevOps field. In a traditional learning approach, they encounter complex tasks that require the students to analyse situations, explore new approaches, and make independent decisions. However, the use of AI can reduce this complexity by automatically simplifying processes and reducing the cognitive load required. This raises the question of whether AI will contribute to improving the quality of DevOps professionals’ training, or whether it could have negative consequences, such as a reduction in independence, analytical thinking, and innovation skills.
Therefore, this article examines the challenges of using AI in an educational environment, by using the example of students specialising in DevOps. The study focuses on analysing the impact of AI on their motivation, academic performance, independence, and the development of key skills essential for professional activities.
The role of artificial intelligence in education is one of the key topics in modern research in pedagogy, cognitive science and learning technologies (Ambarita, N., & Nurrahmatullah, M. F., 2024; Chen, Y., Jensen, S., Albert, L. J., Gupta, S., & Lee, T., 2023; Chen, L., Chen, P., & Lin, Z., 2020). Numerous academic studies highlight the positive aspects of integrating AI into educational environments, including its ability to improve individual student outcomes and provide personalised learning experiences (Sasikala, P., & Ravichandran, R., 2024; Kelly, A., Sullivan, M., & Strampel, K., 2023; Chassignol, M., Khoroshavin, A., Klimova, A., & Bilyatdinova, A., 2018).
Research shows that AI-based adaptive learning systems contribute to better retention of material, especially for students demonstrating average and lower academic performance (Gligorea, I., Cioca, M., Oancea, R., Gorski, A. T., Gorski, H., & Tudorache, P., 2023). These systems analyse students’ mistakes and adapt learning tasks to individual needs (Kabudi, T., Pappas, I., & Olsen, D. H., 2021). Studies confirm that the use of AI can reduce the time needed to master new material and improve the efficiency of exam preparation (Pardosi, V. B. A., Xu, S., Umurohmi, U., Nurdiana, N., & Sabur, F., 2024).
Another important aspect is the automation of the assessment process, which allows teachers to spend more time with individual students (Hooda, M., Rana, C., Dahiya, O., Rizwan, A., & Hossain, M. S., 2022; González-Calatayud, V., Prendes-Espinosa, P., & Roig-Vila, R., 2021). AI helps to reduce subjectivity in grading, which is particularly important in large groups of students (Bagunaid, W., Chilamkurti, N., & Veeraraghavan, P., 2022).
However, alongside the benefits of AI in education, researchers also highlight some challenges. In particular, a study by McMahan and Parsons (2023) points to the risk of ‘passive learning’, where students rely on pre-defined solutions provided by AI instead of attempting to independently analyse and solve problems.
Whereas, high-achieving students tend to analyse material more deeply, seek alternative solutions, and show initiative in their learning (Mašková, I., Kučera, D., & Nohavová, A., 2024).
However, according to a study by Nizamani, Ramzan, Fatima, and Asif (2024), the use of AI can increase emotional stress and dependency, thus highlighting the need for AI systems that balance cognitive enhancement with emotional well-being. Students who rely on AI for quick problem solving may lose motivation to independently analyse complex concepts (Chen, C., Hu, W., & Wei, X., 2024).
Thus, the literature review shows that AI has both positive and negative impacts on the educational process. The main risk lies in the potential development of passive learning, especially among high-achieving students. This poses a challenge for researchers to develop AI-driven approaches that maintain learning motivation and foster critical thinking skills.
The aim of this study is to analyse the impact of artificial intelligence on the academic performance of students specialising in DevOps, and their ability to learn independently. A particular focus is on determining whether AI promotes the development of critical thinking and professional competencies in DevOps specialists or, conversely, reduces cognitive effort, which could affect the quality of their education.
Modern AI-driven educational platforms can both support students in their learning and create dependency on automated solutions. It is crucial to determine how these technologies affect students studying DevOps: do they stimulate further development, or do they create conditions that reduce independent thinking and initiative?
The study will address the following key questions:
Therefore, this study aims to identify both the positive and potentially negative consequences of AI adoption in the education of DevOps students. The findings can be used as a basis for developing more effective teaching methods that encourage independent thinking, innovative problem solving and analytical skills, which are essential for professional success in the DevOps field.
This study adopts a mixed-methods approach, combining both quantitative and qualitative analysis with the objective to comprehensively investigate the impact of artificial intelligence on the learning process of students specialising in DevOps. The research consisted of several stages, including a literature review, a student survey, and an experimental study with control and experimental groups.
The first stage involved a systematic review of scientific publications related to the impact of AI on the learning process, student motivation, and the development of critical thinking in DevOps specialists.
A systematic review of scientific publications was conducted in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. The search period was the years 2020–2024. The following databases were investigated: Scopus, Web of Science, IEEE Xplore, SpringerLink, and Google Scholar. Keywords and logical combinations were inserted as follows: (“artificial intelligence” OR “AI”) AND (“education” OR “learning”) AND (“DevOps” OR “software engineering”) AND (“critical thinking” OR “self-directed learning” OR “motivation”).
The study involved 120 third- and fourth-year Information Systems and Technologies students at Lviv Polytechnic National University. To ensure the reliability of the results, the participants were assigned to groups by using a randomisation procedure.
Initially, all participants completed an entrance diagnostic test comprising 15 tasks on fundamental DevOps subjects (CI/CD, containerisation, monitoring and cloud services) and 10 questions designed to evaluate their critical thinking skills (an adapted section of the Watson–Glaser test) (see Appendix A).
The results made it possible to evaluate each student’s level of prior knowledge and cognitive skills (Ipre).
Randomisation was performed by using a random number generator (specifically, the RAND() function in Microsoft Excel).
Each student was assigned a unique code, after which, they were randomly divided into two equal groups of 60: An Experimental Group (EG) received training with the use of AI, while a Control Group (CG) received training without the use of AI.
To verify the equivalence of the groups, a statistical comparison of the entrance test results was performed by using Student’s t-test for independent samples.
The difference in average scores between the two groups was not statistically significant (p = 0.27 > 0.05), which indicates homogeneity in terms of the level of preparation prior to the experiment.
Therefore, any differences in results after the experiment can be attributed to the use of AI in the learning process rather than to the initial differences in the students’ knowledge or cognitive skills.
The main stage of the research involved completing a series of tasks. These tasks are summarised in Table 1; they included automating CI/CD processes (configuring Jenkins/GitHub Actions), optimising containerisation (working with Docker/Kubernetes), and configuring monitoring and logging (Prometheus/Grafana).
|
№ |
Task type |
Brief description |
Assessment criteria |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
During the experiment, students in the experimental group used ChatGPT 4 (OpenAI’s model from March 2024) as their primary generative artificial intelligence tool.
They used the model to obtain explanations of DevOps concepts such as CI/CD, containerisation, monitoring and infrastructure as code, to generate configuration file templates such as Dockerfile, docker-compose.yml and .gitlab-ci.yml, to analyse logs and search for errors in scripts, and to generate technical solution options while implementing laboratory tasks.
ChatGPT 4 usage rules were the following:
After completing the tasks, the students took a test to assess their understanding of the material and the development of their critical thinking skills (see Appendix B). A survey was also administered to assess their motivation for independent learning after using AI.
The overall level of DevOps material assimilation, critical thinking development and independent learning was calculated for each participant by using an integral Ipost indicator.
Thus, based on this integral indicator, the following was performed:
The study was conducted in accordance with the ethical standards of the academic community. All participants were informed about the aim and objectives of the experiment, and all participants gave their explicit consent to participate. All data were processed anonymously, and the results were used for scientific purposes only.
The results obtained indicate an ambiguous impact of artificial intelligence on high-achieving students, especially in the context of DevOps education. The analysis of the survey and experimental results suggests that AI improves the learning efficiency of students with medium and low levels of preparation, but may negatively affect the development of independent thinking and initiative of high-achieving students.
The survey found that:
To compare the results of the control (CG) and experimental (EG) groups after the pedagogical experiment, the mean values (M) and standard deviations (SD) were calculated for indicators reflecting assimilation of the educational material, critical thinking development, independence development, and behavioural aspects when using ChatGPT 4 for DevOps tasks.
As shown in Table 2, the data reveal general trends in the improvement of the experimental group’s performance on most indicators. Notably, there was a significant increase observed in the mastery levels of DevOps material, critical thinking, independence, reflection, and cognitive activity (curiosity). Meanwhile, the indicators of external motivation and cognitive load were lower. This is consistent with the students’ responses in the questionnaire regarding the reduced need for lengthy information searches when using AI.
|
Indicator |
CG (M ± SD) |
EG (M ± SD) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
A statistical analysis using Student’s t-tests for independent samples confirmed statistically significant differences between the control (CG) and experimental (EG) groups for most of the studied indicators. The significance level was set at p < 0.05. Specific aspects yielded the following data points:
DevOps Material Assimilation (DevOps Score): The results showed that students in the experimental group had significantly higher average values (M = 11.53, SD = 2.01) than those in the control group (M = 8.08, SD = 2.09). A t-test revealed a large effect of using ChatGPT 4 on the quality of DevOps material assimilation (t(118) = 7.68, p < 0.001, Cohen’s d = 1.35).
Critical Thinking Score: The scores for the experimental group (M = 8.37, SD = 1.41) were significantly higher than those for the control group (M = 5.98, SD = 1.69). The T-test result was t(118) = 6.12, p < 0.001, and Cohen’s d was 1.12. These results suggest that students significantly improved their ability to analyse, argue, and evaluate AI recommendations when performing DevOps tasks.
Self-independence index (SelfIndex): Self-independence indicators increased sharply after the experiment, from M = 3.06, SD = 0.49 in the control group (CG) to M = 4.05, SD = 0.49 in the experimental group (EG). A t-test revealed a significant difference between the two groups (t(118) = 8.93, p < 0.001, Cohen’s d = 1.48). These results suggest that ChatGPT 4 not only facilitated better assimilation of the learning material but also enhanced the students’ sense of independence.
Motivational and behavioural parameters. Motivation: Scores decreased in the experimental group (EG) (M = 2.70, SD = 0.77) compared to the control group (CG) (M = 3.90, SD = 0.68), t(118) = –8.18, p < 0.001. This indicates a decrease in external motivation when accessing AI, according to 65% of the questionnaire responses. Cognitive load: Lower than average in the EG (M = 3.08) compared to the CG (M = 3.82), t(118) = –5.23, p < 0.001. This indicates a decrease in the cognitive load due to automated prompts. Reflection: A significant increase in the EG (M = 3.97) compared to the CG (M = 2.57), t(118) = 9.06, p < 0.001, d = 1.65. This demonstrates the development of reflective competence. Curiosity: There was an increase in the EG (M = 4.12) compared to the CG (M = 2.97). This difference was significant, at t(118) = 7.91, p < 0.001, d = 1.43. Thus, ChatGPT 4 positively affected the students’ cognitive activity and research interest.
Completing the tasks related to CI/CD process automation and containerised environment management showed the following results:
DevOps engineers must possess strong independent analysis and critical thinking skills because their work involves integrating and automating complex information systems. In this context, artificial intelligence has a dual effect. On the one hand, AI promotes the rapid assimilation of DevOps basics by providing students with ready-made solutions, explanations, and practical applications of technology. However, over-reliance on AI can reduce the depth of the understanding of processes if students merely copy recommendations without critically analysing them. Additionally, regular use of automated prompts creates a dependency on AI tools, which can negatively impact one’s ability to work in situations without ready-made solutions, such as debugging non-standard DevOps pipelines or troubleshooting complex technical errors.
The results of the study show that the impact of AI on the learning process is heterogeneous and varies according to students’ ability levels. In particular, students with lower academic performance benefit significantly from the use of AI in their studies, while high-achieving students may lose motivation for independent analysis. This is particularly relevant in the context of DevOps education, where a deep understanding of complex systems and the ability to solve non-standard tasks independently are crucial. Table 3 summarises the main positive and negative aspects of the impact of AI on different categories of students.
|
Category of Students |
Positive Impact of AI |
Negative Impact of AI |
|
|
|
|
|
|
|
|
|
|
|
|
The results obtained suggest that AI is an effective tool for supporting learning, but its use requires limitations and a conscious approach. In particular, it is important to maintain a balance between automated guidance and the development of students’ analytical skills.
AI exhibits a positive impact on medium- and low-ability students, by helping them to grasp material more quickly and improve their academic performance. However, its use can hinder the development of high-achieving students by reducing their motivation for independent analysis and the pursuit of unconventional solutions.
Students who actively use AI often arrive at correct answers, but they do not always understand the underlying mechanisms. This is particularly important in the field of DevOps, where it is not only important to know the end result, but also to develop and possess a deep understanding of automation processes, CI/CD pipelines, and software architecture.
The group of students who worked without AI showed a higher level of cognitive development, thereby confirming the importance of traditional learning methods in preparing highly qualified DevOps professionals. This highlights the need for a balanced approach to integrating AI into the educational process, while ensuring that the technology supports the development of independent thinking instead of replacing it.
After completing the tasks, several factors were analysed, including work speed, the depth of understanding of the material, motivation levels and the ability to deal with unconventional situations. One week later, the students took a follow-up test to assess long-term retention. Table 4 summarises the results obtained.
As can be seen in the table, the use of AI provided short-term benefits in terms of the speed of task completion, but reduced independent thinking and long-term retention of material.
For DevOps education, these findings are critical, as the field requires not only the ability to find solutions quickly, but also to acquire a deep understanding of system architecture, the ability to deal with unconventional problems, and the foresight to anticipate the consequences of changes in the software environment.
|
Indicator |
AI Group |
Non-AI Group |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Therefore, new approaches towards integrating AI into DevOps education should be developed with the objective to maintain a high level of student engagement in independent analysis and creative thinking.
The results obtained are consistent with the Cognitive Load Theory, which states that the learning effectiveness depends on an optimal balance of external, internal, and relevant loads (Sweller, 2011). Usage of ChatGPT 4 to perform DevOps tasks reduced the students’ external cognitive load because the model provided quick access to explanations and code examples. This enabled students to allocate more mental resources to the relevant load of understanding the logic of CI/CD processes, containerisation, and system monitoring. The decrease in cognitive load indicators in the experimental group (M = 3.08 vs. 3.82 in the control group) confirms that AI contributed to an increased productivity in knowledge acquisition. However, excessive simplification of tasks could reduce depth of understanding, which aligns with Sweller’s warnings about the risk of ‘assistance overload’.
The interpretation of the results is consistent with the Self-Determination Theory, which posits that motivation stems from three basic psychological needs: autonomy, competence, and social relatedness (Evans, P., Vansteenkiste, M., Parker, P., Kingsford-Smith, A., & Zhou, S., 2024). When using ChatGPT 4, students’ sense of competence increased as they received quick explanations and examples, boosting their confidence in their knowledge. However, the survey showed that 65% of the students surveyed reported decreased motivation to find solutions independently, thereby indicating a reduction in their need for autonomy. This reflects a common AI-related dilemma: technology can support learning effectiveness but also limit self-regulation development if its use is not accompanied by pedagogical control.
Thus, the results of the experiment can be interpreted as striking a balance between reducing cognitive load and partially weakening autonomous motivation. According to the Cognitive Load Theory, optimising external assistance promotes better knowledge acquisition. However, the Self-Determination Theory emphasises that sustained learning motivation is only formed when students retain control over the learning process. Therefore, the introduction of ChatGPT 4 into DevOps-oriented training should be accompanied by the development of metacognitive strategies, reflection, and conscious use of AI to support cognitive efficiency and intrinsic motivation to learn.
The results of the study confirm that the integration of artificial intelligence into the educational environment has a dual impact: while it can significantly improve learning outcomes for average-level and struggling students, it also poses risks to the development of independent thinking among high-achievers. This is particularly relevant in the field of DevOps, where a profound understanding of complex processes is required, rather than simply applying off-the-shelf solutions.
AI is a powerful learning tool that can significantly accelerate task completion. However, it also reduces the cognitive load, which can have a negative impact on the development of critical thinking, particularly for students seeking a deeper understanding of the material.
High-achieving students who actively use AI tend to be less motivated to solve complex problems independently. This is because the technology provides quick and accurate answers, thus reducing the need for self-analysis and thoughtful problem solving.
In the context of DevOps education, reliance on AI can be a significant challenge, as the field requires not only process automation, but also the ability to think outside the box, troubleshoot technical issues, and develop optimal solutions.
The results of the experiment show that those students who studied without actively using AI demonstrated superior understanding of the material and achieved better results in the long term. This highlights the importance of a balanced approach to integrating AI into the educational process, ensuring that the technology enhances learning instead of merely replacing essential cognitive skills.
Rather than restricting or banning the use of AI altogether, the approach to learning should be adapted to focus more on developing analytical and creative thinking. For high-achieving students, an effective solution would be to introduce tasks that encourage deep analysis and unconventional approaches to problem solving.
One such approach is the use of open-ended project tasks where there is no single correct answer. This encourages students to explore multiple possible solutions, evaluate their advantages and disadvantages, and choose the optimal option based on data analysis.
A valuable tool can be scenario-based cases that simulate real-world DevOps processes, where artificial intelligence is used as a supporting tool but does not provide a ready-made solution. This allows students to develop critical thinking skills and learn to make decisions under uncertain conditions.
In addition, it is essential to incorporate group discussions where students must justify their decisions based on their own experience and knowledge, rather than relying solely on AI-generated answers. This approach fosters teamwork, communication, and analytical thinking skills – which are key competencies for professionals in the DevOps field.
To prevent passive learning, it is advisable to implement a ‘justified use of AI’ methodology, which requires students to first independently analyse a problem and propose their own approach before seeking AI-generated guidance.
In the context of DevOps education, this approach could involve documenting an initial problem-solving plan before consulting artificial intelligence. This practice helps students structure their thoughts and develop logical thinking skills.
After receiving an AI-generated answer, it is important to critically evaluate it and explain which aspects have been modified or added. This approach promotes a more extensive understanding of the material, and enhances the analytical skills needed to work effectively in a DevOps environment.
Instead of passively copying AI-generated answers, students should be trained to use AI as a tool for hypothesis testing, error analysis, and solution optimisation. In DevOps, this could mean using AI for log analysis and monitoring to identify problems and make informed decisions, rather than simply automatically generating codes or configurations.
It could also mean implementing automated process optimisation recommendations that require manual validation before application. This approach fosters critical thinking and analytical evaluation skills, which are essential for working effectively in a DevOps environment.
The use of AI in education, particularly in DevOps, remains an open question that requires further research. It is important to investigate the long-term impact of AI on students’ cognitive development, in particular, whether its continuous use leads to a decline in independent thinking.
Particular attention should be paid to finding optimal ways to integrate AI into DevOps education, while ensuring a right balance between process automation and the need to train students to independently analyse complex systems.
A promising research direction is the development of adaptive AI algorithms that support the growth of students’ analytical and creative thinking, rather than simply providing ready-made solutions. This would help to maintain motivation for learning and promote a deeper understanding of DevOps processes.
All of the educational activities described in this article were implemented in accordance with Lviv Polytechnic National University’s ethical standards. Data collection and analysis were carried out in accordance with the principles of academic integrity and respect for the dignity of participants in the educational process, as well as the ethical treatment of human research subjects.
The tests, student projects and related educational activities described in the article were officially approved by the Department of Information Systems and Networks at Lviv Polytechnic National University (protocol No. 1, 28 August 2025). As the study did not involve the use of personal or sensitive data, nor did it pose any risk to the participants, an official exemption from the requirement to obtain written informed consent was granted. Nevertheless, all students were informed about their participation in the educational experiment and the purpose of the study, and were given the opportunity to withdraw from participation at any time without facing any negative consequences.
The department confirms that all research procedures comply with the ethical provisions of the Declaration of Helsinki (World Medical Association) and the current international standards that govern research involving human subjects in an academic environment.
Mykhailo Luchkevych: conceptualization, data curation, formal analysis, validation, visualization, writing – original draft.
Viktoriia Luchkevych: conceptualization, methodology, investigation, resources, writing – review and editing.
References
Ambarita, N., & Nurrahmatullah, M. F. (2024). Impacts of artificial intelligence on student learning: A systematic literature review. Jurnal Varidika, 36(1), 13–30. https://doi.org/10.23917/varidika.v36i1.4730
Bagunaid, W., Chilamkurti, N., & Veeraraghavan, P. (2022). Aisar: Artificial intelligence-based student assessment and recommendation system for e-learning in big data. Sustainability, 14(17), Article 10551. https://doi.org/10.3390/su141710551
Chassignol, M., Khoroshavin, A., Klimova, A., & Bilyatdinova, A. (2018). Artificial Intelligence trends in education: a narrative overview. Procedia computer science, 136, 16–24. https://doi.org/10.1016/j.procs.2018.08.233
Chen, C., Hu, W., & Wei, X. (2024). From anxiety to action: exploring the impact of artificial intelligence anxiety and artificial intelligence self-efficacy on motivated learning of undergraduate students. Interactive Learning Environments, 33(4), 3162–3177. https://doi.org/10.1080/10494820.2024.2440877
Chen, L., Chen, P., & Lin, Z. (2020). Artificial intelligence in education: A review. IEEE Access, 8, 75264–75278. https://doi.org/10.1109/ACCESS.2020.2988510
Chen, Y., Jensen, S., Albert, L. J., Gupta, S., & Lee, T. (2023). Artificial intelligence (AI) student assistants in the classroom: Designing chatbots to support student success. Information Systems Frontiers, 25(1), 161–182. https://doi.org/10.1007/s10796-022-10291-4
Evans, P., Vansteenkiste, M., Parker, P., Kingsford-Smith, A., & Zhou, S. (2024). Cognitive load theory and its relationships with motivation: A self-determination theory perspective. Educational Psychology Review, 36(1), Article 7. https://doi.org/10.1007/s10648-023-09841-2
Gligorea, I., Cioca, M., Oancea, R., Gorski, A. T., Gorski, H., & Tudorache, P. (2023). Adaptive learning using artificial intelligence in e-learning: a literature review. Education Sciences, 13(12), Article 1216. https://doi.org/10.3390/educsci13121216
González-Calatayud, V., Prendes-Espinosa, P., & Roig-Vila, R. (2021). Artificial intelligence for student assessment: A systematic review. Applied sciences, 11(12), Article 5467. https://doi.org/10.3390/app11125467
Hooda, M., Rana, C., Dahiya, O., Rizwan, A., & Hossain, M. S. (2022). Artificial intelligence for assessment and feedback to enhance student success in higher education. Mathematical Problems in Engineering, 2022(1), Article 5215722. https://doi.org/10.1155/2022/5215722
Kabudi, T., Pappas, I., & Olsen, D. H. (2021). AI-enabled adaptive learning systems: A systematic mapping of the literature. Computers and Education: Artificial Intelligence, 2, Article 100017. https://doi.org/10.1016/j.caeai.2021.100017
Kelly, A., Sullivan, M., & Strampel, K. (2023). Generative artificial intelligence: University student awareness, experience, and confidence in use across disciplines. Journal of University Teaching and Learning Practice, 20(6). https://doi.org/10.53761/1.20.6.12
Mašková, I., Kučera, D., & Nohavová, A. (2024). Who is really an excellent university student and how to identify them? A development of a comprehensive framework of excellence in higher education. European Journal of Psychology of Education, 39(4), 4329–4363. https://doi.org/10.1007/s10212-024-00865-y
McMahan, T., & Parsons, T. (2023). Active and Passive Machine Learning Predictors to Build Adaptive Virtual Environments. Human Factors in Virtual Environments and Game Design, 96, 47–54. https://doi.org/10.54941/ahfe1003866
Nizamani, M., Ramzan, F., Fatima, M., & Asif, M. (2024). Investigating How Frequent Interactions with AI Technologies Impact Cognitive and Emotional Processes. Bulletin of Business and Economics (BBE), 13(3), 316–325. https://doi.org/10.61506/
Pardosi, V. B. A., Xu, S., Umurohmi, U., Nurdiana, N., & Sabur, F. (2024). Implementation of an Artificial Intelligence Based Learning Management System for Adaptive Learning. Al-fikrah: Jurnal Manajemen Pendidikan, 12(1), 149–161. http://dx.doi.org/10.31958/jaf.v12i1.12548
Sasikala, P., & Ravichandran, R. (2024). Study on the Impact of Artificial Intelligence on Student Learning Outcomes. Journal of Digital Learning and Education, 4(2), 145–155. https://doi.org/10.52562/jdle.v4i2.1234
Sweller, J. (2011). Chapter two – Cognitive load theory. In J. P. Mestre & B. H. Ross (Eds.), Psychology of Learning and Motivation (Vol. 55, pp. 37–76). Academic Press. https://doi.org/10.1016/B978-0-12-387691-1.00002-8
I. Questions on basic DevOps topics (15 questions, 1 point for each correct answer, maximum 15 points).
1. What is the main objective of the Continuous Integration process?
a) Automatic code compilation and testing after each commit
b) Automatic creation of virtual machines
c) Monitoring server load
d) Managing user roles
2. Which GitLab CI file defines the pipeline structure?
a) docker-compose.yml
b) gitlab-ci.yml
c) jenkinsfile
d) pipeline.yaml
3. What is a ‘pipeline stage’?
a) The stage of artefact collection
b) A sequential or parallel stage in the CI/CD process
c) A level of system monitoring
d) A step in container deployment
4. The command ‘docker build -t app.’ performs the following:
a) Creates a Docker network
b) Builds an image based on the Dockerfile
c) Starts a container
d) Deletes a container
5. What is the difference between a container and a virtual machine?
a) A container has its own OS kernel
b) A container shares the OS kernel
c) A virtual machine is lighter than a container
d) There is no difference
6. Which Kubernetes command creates a deployment?
a) kubectl get pods
b) kubectl apply -f deployment.yaml
c) kubectl start service
d) kubectl config view
7. Which statement correctly describes how Prometheus works?
a) It uses a push model for data collection
b) It uses a pull model, polling targets via HTTP
c) It stores data only in JSON
d) It only works with Docker
8. What role does Grafana play in the monitoring system?
a) Generates logs
b) Collects metrics
c) Visualises data and creates dashboards
d) Manages containers
9. What is an ‘alert’ in Prometheus?
a) A compilation error message
b) A mechanism for notifying when specified metric thresholds are exceeded
c) A container update report
d) A deployment process
10. Which of the following belongs to the IaaS model?
a) AWS EC2
b) Google Docs
c) GitHub Actions
d) Slack
11. What does the principle of ‘Infrastructure as Code’ mean?
a) Creating infrastructure using software code
b) Coding user interfaces
c) Describing the business logic of a service
d) Storing code in a database
12. Which technology is most often used for IaC?
a) Terraform
b) Hadoop
c) Jenkins
d) Anaconda
13. Which of these principles corresponds to the DevSecOps approach?
a) Security is only considered at the testing stage
b) Security is integrated at all stages of CI/CD
c) Security is provided only by the administrator
d) Use of separate servers for security
14. What is the purpose of the Jenkins system?
a) Container management
b) Cloud resource monitoring
c) Automation of build and deployment processes
d) Network traffic analysis
15. Which statement best describes the DevOps culture?
a) Separation of developers and administrators
b) Continuous collaboration, automation and shared responsibility
c) Testing only after deployment
d) Complete rejection of documentation
Section I assessment: 0–7 points = low; 8–12 points = medium; 13–15 points = high.
II. Tasks for assessing the overall critical thinking ability (10 questions, 1 point for each correct answer, maximum 10 points).
1. Just because all DevOps engineers work with Git and some of them use CI/CD, does that mean that all Git users apply CI/CD?
a) Yes
b) No
c) Not enough data to give an answer
2. A statement is given: “The more automation, the fewer problems”. What hidden assumption is contained in this conclusion?
a) All errors are caused by human factors
b) Automation is always effective under any conditions
c) Automation can reduce human fatigue
3. If productivity increased by 30% after the introduction of AI, this would mean that:
a) AI has improved all aspects of work
b) There is a connection between AI and productivity, but not necessarily a causal one
c) AI always increases productivity
4. An argument is given: “Using ChatGPT makes students passive”. Evaluate the strength of the argument.
a) Strong, as it is supported by research
b) Weak if there is no empirical evidence
c) Neutral if there are mixed results
5. Three students who used AI achieved higher grades. Conclusion: AI improves everyone’s performance.
a) Correct
b) Incorrect
c) It is impossible to judge based on such a small sample
6. If all Jenkins scripts are stored in Git, some of which are private, can we assume that all Jenkins scripts are private?
a) Yes
b) No
c) Not enough information is given
7. The CI/CD pipeline fails due to an error in the test. You assume that the error is in the code. What error in reasoning could you be making?
a) Confirmation bias
b) Generalisation error
c) Causal confusion
8. Two sources provide different advice on optimising Docker images. What is the best course of action?
a) Choose the first link from Google
b) Test experimentally and evaluate the results
c) Apply the solution with the most appealing visualisation
9. A statement is given: “All successful companies use DevOps, so DevOps guarantees success”. Comment on it.
a) Logical fallacy: ‘post hoc’
b) Correct conclusion
c) Generalisation error
10. Sequence of arguments:
1. Automation reduces deployment time.
2. Faster deployment reduces team stress.
3. Less stress improves code quality.
Is this logical sequence convincing?
a) Yes, it is completely logical.
b) Partially logical, but it requires empirical confirmation.
c) No, because there is no evidence of a causal relationship.
Section II scoring: 0–4 points: low level of critical thinking; 5–7 points: average level of critical thinking; 8–10 points: high level of critical thinking
III. Interpretation of the overall test results
|
Component |
Maximum points |
Levels of development |
Weight in the overall indicator |
|
DevOps knowledge |
|
|
|
|
Critical thinking |
|
|
|
Integral index of preliminary preparation (IP):
Interpretation: Ipre ≥ 0.8 – high level of preparedness; 0.5–0.79 – average; <0.5 – low.
I. DevOps material test (maximum 14 points)
Purpose: To assess the students’ understanding of the principles of CI/CD, containerisation, monitoring and cloud services following completion of practical tasks.
Format: Twelve multiple-choice questions with one correct answer and two open-ended questions requiring short written answers.
1. The main goal of CI/CD is:
a) to create cloud servers
b) continuous code integration
c) user management
d) monitoring
2. Which file in Jenkins describes the pipeline?
a) Dockerfile
b) pipeline.yaml
c) Jenkinsfile
d) ci.yml
3. The command ‘docker run -d -p 8080:80 nginx’ performs:
a) image assembly
b) launches a container in the background
c) image update
d) container deletion
4. What is an ‘artifact’ in CI/CD?
a) any compiled result
b) a Docker container
c) an intermediate log
d) a repository branch
5. Which command creates a deployment in Kubernetes?
a) kubectl apply -f deployment.yaml
b) kubectl get pods
c) kubectl config set
d) kubectl expose
6. What role does Helm play in DevOps processes?
a) System monitoring
b) Kubernetes application package management
c) Container testing
d) Access control
7. Prometheus collects metrics by using the:
a) push
b) pull
c) stream
d) query
8. What data format does Prometheus use to store metrics?
a) SQL
b) Time Series
c) JSON schema
d) CSV
9. Grafana is used for:
a) collecting metrics
b) data visualisation and analytics
c) API testing
d) containerisation
10. Which of these is an example of PaaS?
a) AWS EC2
b) Heroku
c) Dropbox
d) VirtualBox
11. What does ‘Infrastructure as Code’ mean?
a) describing infrastructure in the form of a configuration code
b) coding business logic
c) developing the UI
d) storing data in code
12. Which tool is most commonly used for IaC?
a) Terraform
b) Spark
c) SonarQube
d) Git
13. Open question:
Describe the difference between Continuous Integration and Continuous Deployment. The answer should include definitions of both processes, explaining that Continuous Integration involves the automatic collection and testing of code, while Continuous Deployment involves the automatic deployment of the finished version in a production environment.
14. Open-ended question:
Explain how Docker and Kubernetes interact in a typical DevOps pipeline. The description will be evaluated for logical consistency and accuracy in terms of the following steps: container creation, deployment, and cluster orchestration.
Scoring: 1 point awarded for each correct answer to the test questions, and up to 2 points for each open-ended answer. The maximum score is 14 points.
Levels of knowledge: 0–6: low; 7–10: average; 11–14: high.
(adapted from Watson–Glaser; maximum 10 points)
Purpose: To evaluate students’ analytical and logical skills following the use of AI.
1. Does passing all automated tests guarantee that there are no errors in the product?
a) Yes
b) No
c) Only with manual testing
2. You have received two conflicting reports on CI/CD performance. What should you do?
a) Choose the report with the best performance
b) Check the measurement methodology of both reports
c) I gnore the results
3. During deployment, AI recommends removing the testing step. What will you do?
a) Agree
b) Check the consequences first
c) Skip the step
d) Turn off the pipeline completely
4. A student says, “If we automate the process, there will be no more human errors”. This is:
a) a logical conclusion
b) an unfounded assumption
c) an empirical fact
5. Just because three teams increased their efficiency after implementing AI, does that mean that AI is always useful?
a) Yes
b) No
c) Not enough data is given
6. A DevOps engineer says: “If there is an error in the Git repository, it is the system’s fault”. This is:
a) an ad hominem
b) attribution error
c) a fact
d) hypothesis
7. Which of the following statements is an example of correlation rather than causation?
a) More tests = higher quality
b) Larger team size = more bugs
c) Use of IaC = lower costs
8. What is the best way to verify the reliability of an AI advice?
a) Test the result experimentally
b) Accept it without analysis
c) Copy it into the code
9. Conclusion: “All successful DevOps companies use Kubernetes; therefore, Kubernetes ensures success”. This is:
a) Logical
b) A post-hoc fallacy
c) Induction
10. How should we evaluate the argument: “Using AI makes students less creative”?
a) Strong – if supported by evidence
b) Weak – if based on assumptions
c) Neutral
Scoring: 1 point is awarded for each correct answer. Maximum score: 10 points.
Levels: 0–4: low level of critical thinking; 5–7: average; 8–10: high.
Purpose: To determine the impact of AI use on students’ intrinsic motivation and level of independence.
Instructions: Rate each statement on a scale of 1 to 5, where:
‘1’ – ‘strongly disagree’; ‘2’ – ‘disagree’; ‘3’ – ‘unsure’; ‘4’ – ‘agree’; ‘5’ – ‘strongly agree’.
1. Working with AI has made me more motivated to solve technical problems on my own.
2. I analyse AI suggestions before implementing them.
3. Using AI helps me organise my knowledge more effectively.
4. I review my decisions more frequently after consulting AI.
5. I rely too much on AI’s automatic suggestions.
6. I feel that I have become more independent in my learning.
7. I am interested in exploring alternative solutions independently.
8. I complete tasks faster, but with less analysis.
9. AI has boosted my confidence in my own knowledge.
10. I am eager to experiment and deepen my understanding of DevOps technologies.
Results processing:
Positive statements (1, 2, 3, 4, 6, 7, 9 and 10) are evaluated directly.
Negative statements (5 and 8) are inverted.
The average indicator, the independence index, is then calculated.
Interpretation: 4.1–5.0 – high level of independence; 3.0–4.0 – average; below 3.0 – low.
Calculation of the total score after completing DevOps tasks
After the experiment had been finished, the results of each student were evaluated according to three interrelated parameters.
1. Understanding of DevOps material (D): a test of professional knowledge (0–14 points).
2. Critical thinking level (C): an adapted Watson–Glaser test (0–10 points).
3. Independence Index (S): average score on the questionnaire (1–5 points).
To ensure comparability between the scales, all indicators were normalised to a range of 0–1.
where S = 1 corresponds to the minimum level of independence, and S = 5 corresponds to the maximum.
Next, the post-task learning effectiveness indicator (Ipost) was calculated by combining cognitive, analytical and motivational-behavioural results with weighting coefficients determined by experts.
Interpretation of IPOST scores: 0.80–1.00: high level of material assimilation, critical thinking and independence; 0.50–0.79: average level; below 0.50 indicates a low level of skill development.