Test Automation Architecture That Works by Bhupesh Dahal
•Download as PPTX, PDF•
3 likes•1,246 views
The document discusses test automation architecture and provides recommendations for building an effective architecture. It recommends prioritizing unit testing and API/service layer testing over GUI testing to create a testing pyramid. Unit tests should be isolated and test small pieces of code, while API tests can test application logic through service calls. GUI tests should be limited in number and used to test broad end-to-end scenarios, not every small scenario. The goal is to have fewer, more stable automated tests rather than many fragile tests. Following best practices like testing different layers, prioritizing types of tests, and continuous refactoring can help create a maintainable and effective test automation architecture.
Report
Share
Test Automation Architecture That Works by Bhupesh Dahal
- 1. Test Automation Architecture One that works! Presented by Bhupesh Dahal
- 2. Who am I? & Why am I here?
- 3. Agenda • Background - Why test automation, common view of test automation? • Define Problems - Issues most organizations face with test automation. • Fixing the above issues with architecture that has the most ROI. • Review
- 4. Why Test Automation? Dev Team’s Perspective • Increase effectiveness, efficiency, and coverage of software testing. • For regression testing. • Improve quality. Boss’s Perspective • Reduce operational cost. • Increase revenue.
- 5. What comes to your mind when you hear the term “We have Automated Tests”?
- 6. Replicating Manual testing - GUI Testing Functional Test
- 7. Take the weight off of the manual testers
- 8. Focused on GUI testing • Hire consultants onshore/offshore with mantra “Automate everything”. • Qtp, Watir, Selenium Webdriver etc. use tools specific to GUI tests. • Every single small scenario gets added to GUI automation.
- 9. Test starts failing randomly
- 10. When you need the test to run - It never works. Please work!
- 11. What’s the problem eh? • Test execution is slow. • Tests are fragile. • Tests are not reusable. • Tests needs more infrastructure. • Test never works on my box but fails on daily run. More time spent on figuring why the tests are failing. Resource tied up just reviewing failures. Maintenance takes way too much time.
- 13. Goal of Automation Ends up being
- 14. Why do we end up in this situation?
- 16. Golden Rule Testing different layers.
- 17. GUI Layer test • Test the whole application end to end. • Sensitive to UI changes. • Does not require good code. • Works for Legacy system and with newer software architecture like SOA as it is independent of backend technology. e.g. Open a browser, fill in the form and submit the data.
- 18. API/Service Layer test • Test the application logic by making service calls. • Faster than UI testing. • Requires service to exist. • E.g. Instantiate the calculator service and get it to add two numbers.
- 19. Unit level test • Test individual classes. • Much faster than service level testing. • Requires good design.
- 20. Application UI Tightly Coupled Automated UI tests Service LayerAutomated One Level Below UI DAO Layer Fragile Lag with Current Development Robust & Stable With Current Dev cycle Fast Execution UI Independent
- 21. So how to build your pyramid ?
- 22. Start with Unit Test (Quality is not just QA’s responsibility) • Isolation. • Test small piece of code - preferably separate test for each method. • Biggest test should be one test for a class. • Use mock services to remove dependency. • Test only public endpoints. • This forces to use good design.
- 23. Invest more in Service Layer Test • Independent functional test. • Use BDD tools like Cucumber, SpecFlow, Jbehave. • Use mocks but also have specific tests to check third party calls. • Add data validation tests. • Tests for boundary condition. • Tests for error handling - wrong data type, missing data. • Tests around calculations, business rules etc. Gherkin/Plain test DSL(Domain Specific Language) Set up projects/helpers, mock Framework Nunit,Junit,Rspec, xunit GUI Service Layer
- 24. Limit the Number of GUI test • Ask if it can be tested in Service layer before starting. • Start out with use and throw style test. • Gradually build the framework. • Use GUI test as an assistance for manual testing. • Create fewer tests but with broader end to end coverage.
- 25. Quick Guide to Building GUI Test Framework • Do basic meta - programming and start automation. Don’t design the entire framework right away. • Re - factor often. • Review test automation architecture regularly. • Run daily. Gherkin/Plain test Framework Watir/Selenium Webdriver/QTP WorkFlows Pages Navigation UI Utilities
- 26. GUI Service/Api Layer Unit Test Unit Test Business Logic Data access layer Mocked Data access layer Mocked Database Unit tests that isolate dependencies with fakes, mocks, stubs etc Database Can use just unit test framework like Nunit, Xunit prefer BDD style like Cucumber, specflow , Jbehave etc End to end UI test with tools like Watir, Selenium, QTP etc Automated UI Test UI Unit Test Integration Test Integration Test Integration Test Business Logic Business Logic Data access layer Data access layer Data access layer Mocked Database Database Database Data access layer Business Logic
- 27. Once you get the pyramid built
- 28. Review • Prioritize Unit and API/Service test automation. • Limit the number of GUI tests, have a fixed number of stable end-to-end tests. • It’s ok to use multiple technologies. • It’s ok to use Record/Play tools like Selenium IDE for throwaway tests. • It is better to have 50 stable test cases than 500 fragile ones that breaks regularly.
- 29. Hope you all catch tons of Bugs!! https://www.linkedin.com/in/bhupeshdahal
Editor's Notes
- Idea being sold
- We forget or ignore fundamental
- Investing just on GUI test
- Defined by Mike Cohn in succeeding with agile
- Lets go ahead and define each layer
- This provides many advantage of end to end testing without issue of UI tests. The service layer allows for testing business logic at the API or service level, where you're not encumbered by the user interface (UI).
- Summary of above definition
- This provides many advantage of end to end testing without issue of UI tests. The service layer allows for testing business logic at the API or service level, where you're not encumbered by the user interface (UI). DSL for setting values as per business criteria
- Your focus now at the UI level is simply to ensure that the UI itself is working correctly. UI tests are very brittle, so keep these tests to a minimum. These automation tests will need maintenance any time the UI changes, and because there are so many factors that come into play when you run a test that emulates clicks on a screen (such as network speed), such tests can result in false test failures. You can't ignore those test failures, but you don't want to end up spending more time troubleshooting and maintaining UI tests than you spend finding actual code defe