When DATA is the King
The blog talks about recently addressed speaking session at “Agile Testing & Test Automation Summit 2019” in Toronto wherein Anjali Chhabra Nandwani, VP & Head, Digital Assurance Services Americas covered the topic “Application of Machine Learning & Predictive Analytics for Quality Engineering and Continuous Delivery”.
What was really interesting and was one of the key takeaways is the fact that this space is really unexplored in many organizations. “Most of the participants were curious about the use cases and how to implement aspects; but had yet not experienced or implemented any of those in their respective organizations” says Anjali. She adds there are several use cases mainly based on supervised learning paradigm which we, at Tech Mahindra’s Digital Assurance Services team, have implemented for our customers to utilize the power of Machine Learning and further some provide useful predictions on the health and quality of the continuous delivery pipeline as well as aiding the test cycle optimization.
Data is the King
One common question was around the data – how much of data is good enough for the reliability of the algorithms to provide dependable and actionable predictions. While we always say “Data is the king” when it comes to Machine Learning & Predictive Analytics; in the case of application for Quality Engineering, we believe that quality of data is very important while the quantity is also a consideration. As far as the predictions are being used by some SMEs to take calculated decisions (so no auto decision making to start with) we can start with some good quality data for as low as last 15-20 release cycles.
Now the next important aspect is which data to use – that completely depends on the use case – It can range from Requirements Data, to defect data, to production incidents and corresponding remediation data. For example, to effectively predict the risk of production failure due to a change, the machine learning model would require defect data from past test cycles (where the same or similar change was made) as well as production incidents data post that change. The key is to analyze all relevant data sources, clean and process the data and train the model for making the risk predictions. Also, continuously using new data for retraining and realigning the model needs to be considered.
We discussed some other use cases around requirement success predictability, defect de-duplication and intelligent defect triaging that we have implemented – Tech Mahindra’s eConvergence & eAnalytics solutions from the LitmusT platform help in converging data from various data sources (ALM, Jira, ITIL data sources) and applying relevant models based on the use case for predictions. eAnalytics provides you Hotspot analysis (E.g. factors likely to have contributed to the prediction of failure) using explainable AI techniques. Also, based on the hotspot analysis, eAnalytics provides actionable insights; very important feature that helps the SMEs reviewing the predictions to make some right decisions.
Overall, very interesting conversations around this topic and I feel there is so much more organizations can achieve by utilizing data that knowingly or unknowingly gets generated with so many test runs/ delivery pipeline execution and ITIL tools. Hopefully, this area will see more traction and attention from the CIOs, IT directors and QA heads in the times to come.
Glad that we at Tech Mahindra are ready with a tried and tested solution and some key use cases for this – while there are many more use cases that can be defined and implemented as we start uncovering more areas of application.
