Development of the Critical Reflection and Agency in Computing Index

Research Background and Issues

• Problems or Challenges Identified by the Authors: With the growing societal impact of computing technologies, computer science students often lack sufficient awareness and capability to address the ethical and socio-technical implications of their work. Although there have been attempts to integrate ethics into computer science curricula, standardized tools to measure the effectiveness of these efforts are lacking, particularly in assessing students' attitudes toward ethical reflection and social action.
• Importance of the Problem: The current neglect of ethical perspectives in computer science education leads students to perceive ethical issues as "not my job," fostering an indifferent attitude toward ethics. This situation poses challenges to balancing technological development with social responsibility. There is a need to improve educational approaches to cultivate computer science professionals with a strong sense of social responsibility and ethical awareness.
• Research Motivation and Related Work: Existing approaches to computer ethics education often have a narrow theoretical foundation or lack clear, quantifiable evaluation methods. For example, many current interventions focus on ethical discussions and skill development, but the academic community still has limited effective methods to measure changes in students' attitudes.

Solution

• Proposed Methods or Solutions:

  1. Introduced the "Critically Conscious Computing Framework," which explicitly includes three core components: critical reflection, critical agency, and critical action.
  2. Developed the "Critical Reflection and Agency in Computing Index," a questionnaire designed to measure university students' attitudes toward critical reflection and agency.

• Innovative Aspects of the Solution:

  • Proposed a theoretical framework specifically tailored to the computing field, integrating the interplay of social power, ethical values, and technology.
  • Developed a systematic assessment tool guided by practice theory, with initial validation for content and face validity, providing computer science educators and researchers with a tool to quantify students' attitudes.
  • Combined macro-ethics (systemic impacts) and micro-ethics (individual responsibility) perspectives to promote a broad exploration of ethical issues.

• Implementation Steps:

  1. Domain Identification: Reviewed the literature to define and refine the operational constructs of critical reflection and agency.
  2. Item Generation: Generated 45 related questions based on existing research.
  3. Expert Review: Engaged seven experts to review the questionnaire in segments, ensuring it covered the research objectives and that the wording was clear.
  4. Cognitive Interviews: Conducted interactive interviews with students to further validate the questionnaire's acceptability and applicability, optimizing its content.

Research Outcomes

• Specific Outcomes:

  1. Identified "critical reflection" and "critical agency" as two core dimensions and refined their operational measurement methods through literature review and expert evaluation.
  2. Developed a final set of 40 questionnaire items addressing students' understanding of the societal impact of technology, sense of ethical responsibility, and awareness of the relationship between technology and social power.

• Comparison with Existing Solutions and Advantages:

  • Unlike existing methods, this study provides a quantitative, standardized tool rather than relying solely on qualitative analysis or single-case studies.
  • Focuses not only on rapidly enhancing students' ethical reflection abilities but also on fostering their confidence and capability to implement and promote ethical decision-making in professional settings.

• Experimental or Evaluation Results:

  1. Experts unanimously agreed on the theoretical foundation and logical framework of the questionnaire, and their feedback was used to optimize certain question designs.
  2. Cognitive interviews with students confirmed the questionnaire's operability and suitability, addressing issues of unclear wording in the final version.
  3. After multiple rounds of cognitive interviews, the optimized questionnaire demonstrated clear content validity and "face validity" for students.

• Limitations and Future Directions:

  • Limitations: Although the questionnaire has preliminary content and face validity, it has not undergone comprehensive statistical validation, such as factor structure and reliability analysis. Additionally, expert reviews were primarily academic, lacking perspectives from industry practitioners. The cognitive interview sample was drawn from a single institution, necessitating broader validation for the tool's generalizability.
  • Future Directions:
    1. Test and refine the tool across diverse educational and industry contexts.
    2. Combine qualitative interview methods to gain deeper insights into students' ethical awareness changes and behavioral patterns.
    3. Expand the tool to apply to real-world professional environments in computing, enabling it to measure critical awareness among both students and practitioners.

Conclusion

This study contributes to addressing the gap in evaluating tools for higher education in computer ethics, offering a robust theoretical foundation and a validated, practical framework and tool. With further optimization of the index, it has the potential to play a significant role in designing educational programs that cultivate socially responsible and critically aware professionals in the computing field.