Background: Modern DevOps practices enable the rapid delivery of software and services to end users. However, the fast-paced DevOps practices also introduce unprecedented challenges to ensure the quality of the delivered software and services. For example, the rapid release cycles challenge traditional performance engineering: resource- and time-consuming performance/load/stress testing becomes impractical; DevOps practices rely on extensive monitoring data (e.g., logs) to provide continuous feedback from operations to development, whereas monitoring can introduce significant overhead to software.

The Measurement, Observation, and Optimization of Software and its Evolution (MOOSE) lab is in the Department of Computer and Software Engineering, Polytechnique Montréal at Université de Montréal. The mission of MOOSE is to ensure the performance and reliability of intelligent software systems developed and operated with modern DevOps practices. These software systems include classical software systems, AI-based systems, and quantum-based systems. Three activities may contribute to the mission:

• Measurement: Measuring the quality (e.g., performance efficiency) of software systems through statistic estimation (e.g., using performance models) or dynamic analysis (e.g., through performance testing or benchmarking).
• Observation: Monitoring/Observing the runtime behaviors of software systems, including functional and non-functional behaviors (e.g., using logging/tracing). This can be achieved from two mutually complementary perspectives: 1) from the software development’s perspective, improving monitoring code and tools; 2) from the software operation’s perspective, improving the analytics and utilization of monitoring data.
• Optimization: Recommending architecture/design/implementation alternatives to improve the quality of software systems.

Current members of MOOSE are performing research on the following topics:

• Agile Performance Engineering in DevOps
• Cost-effective Monitoring (logging/tracing, log/trace analysis)
• AIOps (Artificial Intelligence for IT Operations)
• MLOps (DevOps for Machine Learning Applications)
• DataOps & Data Quality Engineering
• DevOps for Quantum Software Applications