The Vision of Autonomic Computing: Can LLMs Make It a Reality?

Zhiyang Zhang1*, Fenglin Yu3*, Fangkai Yang2, Xiaoting Qin2, Jue Zhang2†, Qingwei Lin2, Gong Cheng1, Dongmei Zhang2, Saravan Rajmohan2, Qi Zhang2
1 State Key Laboratory for Novel Software Technology, Nanjing University,
2 Microsoft,
3 School of Computer Science, Wuhan University, Wuhan, China
* Work is done during an internship at Microsoft. † Corresponding author.
zhiyangzhang@smail.nju.edu.cn
juezhang@microsoft.com
Paper Code

Abstract

The Vision of Autonomic Computing (ACV), proposed over two decades ago, envisions computing systems that self-manage akin to biological organisms, adapting seamlessly to changing environments. Despite decades of research, achieving ACV remains challenging due to the dynamic and complex nature of modern computing systems. Recent advancements in Large Language Models (LLMs) offer promising solutions to these challenges by leveraging their extensive knowledge, language understanding, and task automation capabilities. This paper explores the feasibility of realizing ACV through an LLM-based multi-agent framework for microservice management. We introduce a five-level taxonomy for autonomous service maintenance and present an online evaluation benchmark based on the Sock Shop microservice demo project to assess our framework's performance. Our findings demonstrate significant progress towards achieving Level 3 autonomy, highlighting the effectiveness of LLMs in detecting and resolving issues within microservice architectures. This study contributes to advancing autonomic computing by pioneering the integration of LLMs into microservice management frameworks, paving the way for more adaptive and self-managing computing systems. The code will be made available at https://aka.ms/ACV-LLM.

Key Contributions

  1. We advance the domain of autonomic computing for microservice management through an LLM-based multi-agent framework. To the best of our knowledge, we are the first research work to explore microservice self-management using LLM-based agents.

  2. We establish a taxonomy consisting of five levels for autonomous service maintenance. We also present an online evaluation benchmark designed to assess tasks corresponding to each level of autonomy within the microservice demo project Sock Shop.

  3. We conduct a rigorous evaluation of our LLM-based microservice management framework using the aforementioned benchmark.

Main Ideas

Hierarchical UCB

Figure 1: Hierarchical service management framework with LLM-based multi-agent design.



Hierarchical UCB

Figure 2: Illustration of the Sock Shop example: Each microservice is converted into an LLM-based autonomic agent, managed by an autonomic layer comprising a Planner and Executor. The high-level group manager employs a Plan-Execute feedback mechanism to generate plans and assign sub-tasks to low-level autonomic agents. A decoupled message queue system serves as the middleware to manage communication, collecting feedback and unresolved issues.



Hierarchical UCB

Figure 3: Taxonomy of autonomous levels in service maintenance, focusing on Self-Healing and Self-Optimization.

Citation

@misc{zhang2024visionautonomiccomputingllms,
      title={The Vision of Autonomic Computing: Can LLMs Make It a Reality?}, 
      author={Zhiyang Zhang and Fangkai Yang and Xiaoting Qin and Jue Zhang and Qingwei Lin and Gong Cheng and Dongmei Zhang and Saravan Rajmohan and Qi Zhang},
      year={2024},
      eprint={2407.14402},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2407.14402}, 
}