Software Quality and Test Strategies for Ruby and Rails Applications
•
6 likes•2,416 views
This document provides an overview of software quality and test strategies for Ruby and Rails applications. It discusses the importance of quality, managing quality through setting goals and measuring metrics. It outlines a test strategy template and covers test types, tools, and approaches for unit, integration, acceptance and other types of tests in Ruby/Rails. It also discusses test data management, defect management, and the Ruby/Rails testing ecosystem including various testing frameworks and quality/metrics tools.
Report
Share
More Related Content
What's hot
What's hot (20)
Similar to Software Quality and Test Strategies for Ruby and Rails Applications
Similar to Software Quality and Test Strategies for Ruby and Rails Applications (20)
More from Bhavin Javia
Recently uploaded
Recently uploaded (20)
Software Quality and Test Strategies for Ruby and Rails Applications
- 1. Software Quality and Test Strategies for Ruby and Rails Applications - Bhavin Javia May 29th, 2011
- 2. About Me Ex ThoughtWorker Agile/RoR Consultant - Ennova, Australia Founder - Ennova, India
- 4. Co-Author Dr Adrian Smith Managing Director - Technology Agile Coach @
- 5. Content Software Quality Test Strategy Ruby/Rails Testing Ecosystem Quality and Testing for Rails Projects
- 8. “Quality is the most important factor for deriving long term value from a software product” - Jim Highsmith (Co-Author of Agile Manifesto)
- 9. ”Continuous attention to quality can help maintain velocity, maintain the ability to deliver, without it the team will slow down” - Robert C. Martin (Author of Clean Code)
- 10. Why worry ? Needs to be built in (implicitly) Needs to be defined (to prioritize vs cost/scope/time) Agile project delivery relies on continuous attention to quality Begins at the code level
- 11. code_quality != "App Quality"
- 12. Internal Quality vs External Quality
- 13. External Quality ( Quality as perceived by users ) interface design performance defects user experience
- 14. Internal Quality ( Quality of the implementation and system architecture ) duplication complexity coupling test coverage
- 15. “Technical Debt may be costing you more than you imagined!” - Jim Highsmith
- 16. The trade-off
- 19. Managing Quality
- 20. Managing Quality Set Goals "Max 3 clicks to any page" Measure "Page X takes 5 clicks" Prioritize "Make page X reachable in <= 3 clicks"
- 21. How to achieve Quality ?
- 22. TEST TEST TEST
- 23. What/When/How to Test ?
- 24. " Test Strategy "
- 26. Test Strategy Template Not a Waterfall style document A template to structure the discussion Prompts for things to consider Flexible format
- 27. Purpose Create a shared understanding of approach tools targets timing of test activities
- 28. Guiding Principles Shared Responsibility Everyone is Responsible for Testing and Quality Test Automation All types of tests should be automated (except exploratory tests) Data Management Production data must be obfuscated before use Test Management Tests, documents and data treated as production code
- 29. Quality and Test Objectives Correctness Integrity Maintainability Availability Interoperability Performance
- 30. Quality and Test Objectives Correctness Features and functions work as intended e.g. 0 Critical Defects Integrity Prevent unauthorized access or information loss e.g. All access via HTTPS, Encrypted passwords Maintainability Easy to add new features or fix defects e.g. Code Complexity < 8, Unit Test Coverage > 80% ...
- 31. Quality and Test Objectives Availability Planned uptime percentage e.g. 99.99 % availability Interoperability Ease of information exchange with other systems e.g. Published & versioned API, Supports - IE 8, FF 3.5, Chrome 11 etc Performance Responsiveness & Scalability of the system e.g. Apdex Score > 0.9, Response Time < 200ms
- 32. Quality and Test Objectives Target Measure Prioritize
- 33. Test Scope Systems and features in and out of scope Test both business processes and the technical solution Specify regions and sub-regions included in testing Identify interfaces with other systems System / feature / integration / region / environment /dependency / service In Scope Things to be tested with automation Things to be tested manually Out of Scope Things we can't test e.g. Third Party systems
- 34. Test Approach Identify the test types Identify the tools Include the timing of execution
- 35. Test Types Unit Functional Integration Acceptance Performance Data Conversion
- 36. Test Approach Identify Tools e.g. rspec, cucumber etc Decide Timing of Execution e.g. dev machine, CI server, after each commit etc
- 38. Test Preparation Use requirements captured in user stories Include acceptance criteria in user story Decide approach to prepare and execute tests
- 39. Example
- 40. Acceptance Test
- 41. Environments Development Unit, Functional, Acceptance tests Integration Continuous Integration - Unit, Functional, Acceptance tests Code Analysis/Metrics Staging For exploratory testing Demos to customers Production Smoke/Sanity tests & Monitoring
- 42. Test Execution steps in preparation for deployment/release Build the system Populate test/reference data Execute automated tests Generate test report/code metrics
- 43. Test Data Management System and user acceptance tests Use subset of production data Performance/volume/stress test Use full size production files Generate large volumes of data
- 44. begin database.load! "Sensitive Production Data" ensure database.obfuscate! "Sensitive Data" end
- 46. Why defects ?
- 48. Defect Management Defects only raised when not fixed immediately Capture conditions and severity Capture steps to reproduce Critical Major Minor Trivial
- 49. Defect LifeCycle Identify Prioritize Assign Severity Analyze Write test to reproduce Resolve Fix code to pass the test Verify Run all tests, verify manually Close
- 53. Test Scope
- 54. Test Approach
- 57. What's different in Ruby/Rails ?
- 59. Advantages Testing built right-in Passionate developers Vibrant open source community Excellent tools support
- 60. So what are our options ?
- 62. There are ... 23 Testing Frameworks 8 Javascript Testing Frameworks 8 Browser Testing Frameworks 8 Distributed Testing Frameworks 6 Object Mocking Frameworks 7 Web Mocking Frameworks 2 Continuous Testing Frameworks 7 Continuous Integration Frameworks 12 Code Metrics Frameworks and many more ...
- 63. raise "OMG !!"
- 64. What to Test ? Anything that could possibly break Public interface of each layer Whole application stack Depends on what else you're testing
- 65. What NOT to test ? Cost of test > 10 * Cost of breakage (very rare) Internal implementation Framework code
- 66. You're doing it wrong if Fixing tests all the time Tests that always fail together Tests that never fail
- 67. What to Test ? Models Controllers Views Helpers Libraries APIs
- 68. Testing Options Models Test::Unit (Ruby 1.8, built-in) MiniTest (Ruby 1.9, built-in, replaces Test::Unit) Unit Tests (ActiveSupport::TestCase) Model Specs (rspec) Controllers Functional Tests (built-in) Controller Specs (rspec)
- 69. Testing Options Views assert_select (built-in) View Specs (rspec) Helpers ActionView::TestCase Helper Specs (rspec)
- 70. Testing Options Routes assert_routing (built-in) Routing specs (rspec) Mailers Unit Tests (built-in) Functional Tests (built-in)
- 71. Testing Options Workflow Integration Tests (built-in) Request Specs (rspec) BDD Cucumber + Capybara Cucumber + Rspec
- 72. Testing Options Performance Performance Tests (built-in) NewRelic
- 73. Testing Helpers Data Setup Fixtures (built-in) factory_girl Machinist
- 74. Testing Helpers Mocks/Stubs Rspec mocks Mocha Flexmock API / Web Services VCR Fakeweb Webmock
- 75. Testing Helpers Speed NullDB Autotest/Infinity Test/Watchr Spork parallel_tests TLB CI cijoe cruisecontrol.rb/goldberg integrity
- 76. Quality/Metrics Test Coverage rcov SimpleCov Code Analysis metric_fu Metrical Rails Best Practices
- 77. Which is the Best ? Even the God(s) Can't Say !!
- 78. Which is the Best for Me ?
- 79. IMHO WHATS_BEST = { :newbie => "Start with built-ins", :intermediate => "Checkout alternatives", :expert => "Venture into the obscure" }
- 80. WHATS_BEST[:god] = "Write your own !"
- 81. References http://ennova.com.au/blog http://www.jimhighsmith.com http://www.mountaingoatsoftware.com/tools http://www.informit.com/articles http://www.agileacademy.com.au http://railsrx.com http://www.bootspring.com/blog
- 82. Questions ? http://sqats-rubyconfindia2011.heroku.com bhavinjavia @bhavinjavia [email protected]