The Future of Testing
•Download as PPTX, PDF•
1 like•217 views
Rapid advancement in digital technologies have accelerated the need to deliver more value to customers, faster. This ever-increasing demand for both speed and quality has put software testing and delivery under tremendous stress. How can you advance your development and testing capabilities to meet digital market demand, today and in the future? Is it only about technology and automation? What about testers—will new practices such as AI save us? Join this web seminar for a discussion with Forrester Research’s Diego Lo Giudice and Perfecto’s Tzvika Shahaf about the future state of testing. Learn what’s next for test and dev teams and the people who drive them. We’ll talk about: -Where the development and testing market is headed and how you compare -Test automation challenges and how to overcome them -What’s next in test automation tools and technologies -Testing and development predictions for 2019
Report
Share
The Future of Testing
- 1. THE FUTURE OF TESTING: TRENDS, TACTICS, AND PREDICTIONS
- 2. TZVIKA SHAHAF • Director of Product Management, Perfecto • Co-Author of Continuous Testing for DevOps Professionals • Working with Fortune 500 enterprises and leading tech companies worldwide helping them optimize their testing strategy and achieve Continuous Testing GUEST SPEAKER: DIEGO LO GIUDICE • Vice President & Principal Analyst, Forrester Research • Focus: Application Development & Delivery • Leading expert on SDLC processes and practices • Areas of interest include: agile development, agile and lean transformations, agile development sourcing strategies and services, agile testing practices and tools, DevOps, and software testing and quality SPEAKERS
- 3. AGENDA • The state of continuous testing, trends, and challenges • Continuous testing & AI – How AI can help close the gaps • What’s Next: Predictions for the future of continuous testing / test automation
- 4. © 2017 FORRESTER. REPRODUCTION PROHIBITED. Testing trends and predictions Diego Lo Giudice, Vice President, Principal Analyst @dlogiudice
- 5. 5© 2017 FORRESTER. REPRODUCTION PROHIBITED. The Combined Force of Digital Transformation & Customer Experience Creates Pressure on Application Delivery Teams To Go Faster Digital Transformation Customer Experience Feature Release Cycle
- 6. 6© 2017 FORRESTER. REPRODUCTION PROHIBITED. Agile+DevOps and Continuos Delivery are increasingly what teams are adopting to be successfull............ and testing is right in the middle of it ! Test ReleaseBuild MonitorIdeate/Plan Developers UsersFeedback loop Delivery Pipeline Insights Data/analytics Continuous SW Delivery
- 7. 7© 2017 FORRESTER. REPRODUCTION PROHIBITED. Where are we with testing today ?
- 8. 8© 2017 FORRESTER. REPRODUCTION PROHIBITED. 8% 9% 20% 22% 25% 26% 26% 28% 30% 31% 31% 31% 33% 44% 51% None of the above Other (please specify) Lack of automation tools and/or infrastructure (e.g., Agile PM tools, automated builds, test… Scaling Agile at the enterprise by replicating success throughout the organization to more teams Connecting existing product portfolio management (PPM) with Agile projects Lack of Agile testing skills (e.g., unit testing, test automation, testing in sprints, exploratory… Lack of dedicated cross functional teams Lack of Agile skills in downstream or DevOps (XP, development, testing, continuous integration,… Lack of Agile executive leadership Incompatibility of our existing funding process to support Agile projects Lack of Agile skills in project management practices, backlog grooming, planning, requirements,… Lack of Product Owners commitment from the business Incompatibility of our existing budgeting process to support Agile projects Lack of skilled Product Owners from the business People behavioral change (true buy-in into lean and agile values and principles) Which of the following are barriers to your organization’s Agile adoption? (Check all that apply) Base: 232 professionals with knowledge of their firm’s Agile practices Source: Forrester’s Q3 2017 Global Agile Software Online Survey Testing appears in many of the barriers and challenges in scaling Agile and DevOps Skills Skills Automation
- 9. 9© 2017 FORRESTER. REPRODUCTION PROHIBITED. Clients of leading continuous testing automation tool vendors: levels of automation are over all the place and generally low… CapGemini World Quality Report 2018 shows below 20% for all.
- 10. 10© 2017 FORRESTER. REPRODUCTION PROHIBITED. Test automation is not growing or improving….
- 11. 11© 2016 Forrester Research, Inc. Reproduction Prohibited Performance matters, what broke !....why ? 36% of my Forrester inquiries are on non functional testing - 2/3rds are on performance testing.
- 12. 12 Device Fragmentation & Various OS Platforms
- 13. 13© 2017 FORRESTER. REPRODUCTION PROHIBITED. Testing If you thought Test Automation was hard! Think how hard it is in continuous delivery at speed!
- 14. 14© 2017 FORRESTER. REPRODUCTION PROHIBITED. Can AI help improve software testing ? Source: Forrester - How AI will change software development and applications
- 15. 15© 2017 FORRESTER. REPRODUCTION PROHIBITED. Where can AI play in the extended SDLC and specifically in Testing ? Source Forrester : How AI Will Change Software Development And Applications ?
- 16. 16© 2017 FORRESTER. REPRODUCTION PROHIBITED. 37% use AI and ML to test better and faster to increase quality…What are they doing ? Source Forrester : To Be Published soon “The Path To Autonomous Testing: Augment Human Testers First.”
- 17. 17© 2017 FORRESTER. REPRODUCTION PROHIBITED. Source Forrester: Augmented Intelligence Unlocks The Intelligence In AI
- 18. 18© 2017 FORRESTER. REPRODUCTION PROHIBITED. AI can augment testers intelligence ? Developers Technical testers Business testers AI
- 19. FORRESTER.COM Thank you © 2017 FORRESTER. REPRODUCTION PROHIBITED. Diego Lo Giudice [email protected]
- 20. CONTINUOUS TESTING HAS NEVER BEEN MORE COMPLEX Augmented RealityVirtual Reality Progressive Web Artificial Intelligence Mobile IoT CONTINUOUS TESTING FOR DIGITAL
- 21. Reporting Creation Execution Lab Tools setup Create (Authoring & maintenance) Debugging Execution Scheduling Validations Orchestration Test environment Desktop web Artifacts collection Mobile devicesMobile Emulators Triaging Dashboards Analytics MARKET TREND – AI IN TESTING
- 22. AI-Based Self-Healing: • Network Connectivity • Device Lock • Effective Reboot LEVERAGING AI TO OPTIMIZE AUTOMATION – LAB
- 23. Easy test automation creation & maintenance • Robust & intelligent object addressing • Self-healing executions Powerful web-based IDE • No coding skills required • Script debugging & low maintenance Test management functionality • Manage script/suites and prioritize (if not using CI) LEVERAGING AI FOR AUTOMATION CREATION
- 24. Codeless Selenium LEVERAGING AI FOR AUTOMATION CREATION
- 25. Reasons for “not stable” 80% of issues have a pattern52% success rate 10% of devices, causing 80% of lab issues Lab 25% Orchest ration 25% Scripts & FW 50% FAILURE REASON Objects Coding Time Other Scripts & FW issues Device in use No Device Orchestration issues Networking Stability Lock Other Lab issues
- 26. Reasons for “not stable” 80% of issues have a pattern52% success rate Lab 25% Orchest ration 25% Scripts & FW 50% FAILURE REASON Objects Coding Time Other Scripts & FW issues Device in use No Device Orchestration issues Networking Stability Lock Other Lab issues What’s wrong With my Scripts What’s wrong With my Lab What’s wrong With my Executions 10% of devices, causing 80% of lab issues 0% 10% 20% 30% 40% Issues
- 27. Recognize objects Transcribe speech – Accessibility Make quality related decisions based on data Identify Trends and/or patterns Security use cases – Identify signatures e.g. 27© 2015, Perfecto Mobile Ltd. All Rights Reserved. KEY ML USE CASES IN TEST AUTOMATION
- 28. CarCat Train Glass Dog Sea Sun Table Tree TRAINING AI – THE PHOTO LINGO
- 29. PopupApp not installed Elementnotfound App didn’t launch Capability not supported App Crashed Keyboard Device in Use Trust TRAINING AI – THE TESTING LINGO
- 30. Stabilize Execution By Eliminating Noise • Pop ups • Home screen • Keyboard open Enhanced Reporting and Analysis TRAINING THE TESTING AI ENGINE IMAGE RECOGNITION
- 31. TRAINING THE TESTING AI ENGINE – IMAGE RECOGNITION AI-BASED AUTO-DETECTION OF POPUPS
- 32. TRAINING THE TESTING AI ENGINE – TEXT ANALYSISAI-BASED PROCESS TO IDENTIFY & EXCLUDE FALSE NEGATIVE TEST REUSLTS
- 33. TRAINING THE TESTING AI ENGINE – IMAGE RECOGNITION AI-BASED AUTO-DETECTION OF POPUPS
- 34. # Category 1 Test Authoring 2 Test Maintenance 3 Test Execution 4 Test Analysis 5 Tools Maturity 6 Available Integrations 7 Req. Test/Dev Skillset 8 Testing Env. 9 Testing Types 10 Supported App Types Test Engineers/Developers Manual Testers (Developers?) Traditional Test Automation Define Manual Flows, BDD Style, etc. Test Code & Reusable Functions (Java, JS, etc.) Define Objects, POM, use Object Spy Structured Page Based Test Steps Define Visual Validations and Assertions Longer Time To Develop, Complex Changes required pro-actively, SCM Supported Configure Env. (TestNG Data Provider), Execute Locally/CI/Cloud Flexible, OSS, Commercial High, Including Guidelines, Doc’s Plenty, OSS, Defect Management, etc. Medium-High IDE API, Load, Functional All Types ML Based Test Automation Record Test Flows (No Coding – in most cases) Groups Reusable Functions Objects Generated On-The-Fly (transparent to user) Flow Based Scenarios Part of Test Authoring Recording/Authoring Shorter Time, Advanced Capabilities Self-Healing/Correction Automatically handled/Object Scoring/Local SCM Controlled Configure Env., Execution Management Built-In, Locally/CI/Cloud Relies on ML/AI Tool Vendor in Most Cases Emerging Technology, Web More Advanced Than Mobile Most capabilities built-in, some exists Low-Medium ML UI Mostly Functional (E2E) & API Mostly Web MINDSET AND WORKFLOW CHANGES IN TEST AUTOMATION
- 35. • Microservices & Cloud • Big Data and Intelligent Testing • Service Virtualization • Codeless Test Authoring (Native) • ML/AI Across DevOps Activities • Testing AI TRENDS AND FUTURE OF DEVOPS (PREDICTIONS)
- 36. Thank You