Not as easy as just update: Survey of System Administrators and Patching Behaviours

Document Title

Not as easy as just update: Survey of System Administrators and Patching Behaviours

Document Information

• Domain: Research on System Administrators' Patching Management and Behaviours
• Keywords: Software updates, system administrators, patch management, testing environment, information retrieval, troubleshooting, behavioural research, socio-technical challenges
• Conference: CHI Conference on Human Factors in Computing Systems (CHI ’24), May 11–16, 2024, Honolulu, HI, USA
• DOI: 10.1145/3613904.3642456

Research Background and Issues

• Issues and Challenges: Software patches are fundamentally designed to enhance system security but can also introduce issues and new failures, disrupting organizational operations. Despite the availability of dedicated testing environments and other mitigation measures, little is known about the prevalence of patch management practices in real-world scenarios. The specific impact of contextual factors, such as organizational size and system type, on patch management behaviours has not been systematically studied.
• Significance: System administrators (sysadmins) are responsible for deciding whether and when to introduce patches, decisions that directly affect organizational security and productivity. Failing to apply patches can lead to security vulnerabilities, while premature deployment may risk system unavailability.
• Motivation and Related Work:
  • Previous research indicates that the patch management process involves complex tasks such as information gathering, collaboration, prioritization, decision-making, and troubleshooting.
  • There is a lack of understanding regarding the prevalence of testing environments and the specific impact of information scarcity and online community participation on patch management.
  • Literature suggests that how system administrators coordinate patch management practices in different organizational contexts requires further exploration.

Research Methods and Design

• Methods: A structured survey was conducted, with data sourced from relevant Reddit subforums, mailing lists (e.g., PatchManagement.org), and others. The patch management behaviours and work contexts of 220 system administrators were analyzed.
• Research Questions:
  1. What practices do system administrators adopt at various stages of patch management, and how prevalent are they?
  2. How do contextual factors (e.g., organizational size, system type) influence patch management practices?
• Organizational Sampling:
  • The survey included administrators from organizations of varying sizes, spanning industries such as technology, education, government, and healthcare.
  • Data analysis utilized the Mann-Whitney test to statistically evaluate frequency scores on Likert scales.

Solutions

• Innovations:
  • Provides a comprehensive perspective on patch management practices across organizations of different sizes and system types.
  • Extends existing research by exploring innovative mechanisms for patch management in resource-constrained environments.
• Implementation Steps:
  • Analyze the seven key stages of patch management (i.e., awareness, prioritization, decision-making, preparation, testing, installation, post-installation).
  • Investigate the impact of different contexts (e.g., organizational size, client/server types) on behaviours.
  • Examine administrators' responses to various issues, such as handling problematic patches and selecting information sources.

Research Outcomes

• Key Findings:
  • The prevalence of testing environments was lower than expected, with only 36% of respondents reporting the use of dedicated testing environments; phased deployment was the primary choice.
  • Decision-making by administrators in large organizations was more influenced by management or patch management policies, whereas administrators in smaller organizations relied more on their team’s judgment.
  • Most administrators chose to troubleshoot problematic patches rather than uninstall them, indicating a preference for resolving issues rather than avoiding them.
  • Large organizations more frequently utilized formal tools, such as phased deployment, while smaller organizations, constrained by resources, used these mechanisms less often.
• Advantages:
  • Provides a comprehensive understanding of real-world patch management, addressing knowledge gaps about testing and problem-solving behaviours.
  • Highlights the value of online communities as information resources, particularly during testing and problem resolution stages.
• Experimental or Evaluation Results:
  • Testing setups in large organizations were more driven by patch policies, while smaller organizations were influenced by resource constraints or experience.
  • Server administrators were more likely to conduct formal testing, whereas client administrators relied more on online community resources.
• Limitations and Future Directions:
  • The survey primarily focused on Western developed countries, leading to an uneven regional distribution; future research should address IT high-growth regions such as Asia.
  • Female respondents were underrepresented (only 6%); future studies should explore the unique work experiences of non-male administrators.
  • Recommends developing new tools to enhance collaboration, leveraging online community data to automatically analyze patch information and provide context-specific recommendations for administrators.

Conclusion

• This study provides rich empirical data on how system administrators handle patches, showcasing the balance between security and practical application across different organizational types.
• We recommend future research to focus on:
  • Improving the efficiency of collaboration between online communities and vendors.
  • Developing key technical methods to shorten the patch testing and deployment cycle.
  • Creating tools and resources tailored to system administrators' actual behaviours to support more efficient patch management practices.