Build the Right Regression Suite with Behavior-Driven Testing

While more and more organizations are practicing agile development methodologies, many have not learned how to “bake in quality” throughout the process. As an agile tester, you are an integral part of the development team—working on requirements, design, implementation, writing automated tests, and testing However, are all team members working together as they should to ensure quality from day one through final delivery? Dena Laterza offers proven tips to help you and your team make the cultural shift to adopt and foster a “quality first” team standard. Gain an understanding of a tester's involvement in test-driven development and behavior-driven development. Take back new ideas on automating tests, working with stakeholders, and becoming a fully informed tester. Learn how to push testing back into development and maximize the value of testers on the team. Take back a plan to get your agile team working together—as a team.

The document discusses the "test pyramid" concept for balancing test suites from unit to end-to-end tests. It provides examples of different types of tests including unit tests, integration tests, UI/end-to-end tests. It also discusses challenges with different types of tests and strategies for addressing those challenges including dependency injection, mocks, and tools like Cucumber, Robolectric, and Pacto. The document seeks feedback on testing approaches and provides additional resources on testing best practices.

This document discusses the deployment of test automation tools within IBM's large services division. It describes how test automation was a major problem area, taking up 1/3 of the net testing services business. The author led a research project to develop a technique called Automating Test Automation (ATA) that used natural language processing and program synthesis to semi-automatically convert English test cases into scripts. This helped lower the costs of test automation creation and maintenance. The document then briefly mentions the challenges of selling this research internally to the services division.

We all know that automation is one of the key enablers for those on the CI-CD journey. Most teams are: • implementing automation • talking about its benefits • up-skilling themselves • talking about tooling • etc. In my experience, unfortunately most of the functional automation that is built is: • not optimal • not fit-for-purpose • does not run fast enough • gives inconsistent feedback, hence unreliable Hence, for the amount of effort invested in implementing automation, are you really getting the value from this activity? In this talk, we will discuss these challenges and why it would lead to poor ROI of automation. More importantly, we will discuss the following techniques to make automation valuable: • know the objective for the automation framework • establish criteria for tests to be automated • design your framework with proper abstraction layers • develop using appropriate design patterns

A/B Testing If you are not familiar yet, an introduction to A/B testing and how you can leverage this approach to truly measure customer impact before and after a change. It's a practice highly leveraged in the e-commerce and cloud space to truly measure the impact of a change and be able to iterate through it till you see the desired outcome. Where in your stack to invest in test automation This short talk will explain in which layer to invest in test automation and the pros and cons. Too many teams still invest heavily in automated UI testing which then results in large test automation suites once the platform grows while still not being able to catch critical quality issues before they reach customers. Testing for reliability, resilience and recovery Your customer experience is also impacted by how reliable your application is. How do you test for reliability. But also how do you build and test for resilience, as guaranteed reliability is unachievable and the closer you get the costlier it becomes. Lastly how do you test for recovery, so once an outage or partial outage happened how to you recover, and how do you prepare for that recovery.

Design documentation aimed at engineers or startup founders for how you can create an A/B testing framework in your language/MVC framework of choice.

This document discusses strategies for scaling test automation. It begins with an introduction of the speakers and an overview of continuous testing and the need for scale. It then covers cross-browser testing, the mobile landscape, and how to mix technologies like Selenium, headless browsers, and codeless testing for maximum coverage. The document compares tools and practices, and provides best practices for scaling test automation, including strategies for test data, certification of test scenarios, and measuring return on investment. It concludes with a live demo of scaling Selenium tests on Perfecto.

This document discusses test automation quirks and workarounds for different browsers and devices. It provides examples of common problems such as elements not being interactable, unexpected keyboard inputs, and slow screenshot times. For each problem, it offers a workaround such as adding scrolling before clicks, using custom keys, limiting screenshots, and designing tests to be flexible across environments. The goal is to discuss methods for increasing test code reuse and learning from experiences with different automation platforms and browsers.

SQA job postings are still in abundance, but it is rare to find one that does not include some form of test automation pedigree. Brett will present the topic and then lead the discussion as we explore the various paths to building your test automation acumen, and learn how to add this valuable skill-set to your resume. If you are already an SQA with test automation experience we encourage you to participate and bring your learning forward and into the discussion where we will compare and contrast Computer Science degrees, Code Camps, licensed automation tools such as HP UFT (QTP), test frameworks and scripting tools such as jMeter and SOAPUI. There is much to explore on this topic and we want everyone to leave with a few key areas they can start building on today.

This document discusses test automation in agile projects. It begins with an overview of agile principles like the agile manifesto. It then discusses agile testing principles and practices like continuous integration and continuous delivery. The bulk of the document focuses on test automation, including why it's important, different types of test automation frameworks, and design considerations like the test automation pyramid. It provides tips for test automation including design patterns, abstraction layers, and evolving the framework over time.

Advanced Structures India was established with a motive to address the rising demand for better engineered products in India, particularly in Passenger, Commercial and Agricultural Vehicles segment

Through the webinar, she will give an introduction to the user story concept. How to create them? How they can help us build better products for our customers. Do's and Don'ts.

The document discusses strategic directions for functional test automation. It outlines the history of automation including record and playback, scripting, and table-driven approaches. It then discusses how automation frameworks can address issues like high maintenance costs and specialized skills required. The document shares an example project where an automation framework was used successfully to automate testing for a bank migration project. It achieved quicker test design and the ability for both testers and developers to work with the same test format.

The document discusses using Applitools for visual AI testing. It provides an overview of Applitools, which allows testing user interfaces through visual assertions rather than traditional code-based assertions. This avoids issues like managing locators and adjusting assertions. The document then demonstrates how to set up and implement visual AI testing using Applitools in Java, including configuring Eyes, setting up tests, and executing visual checks on elements and full pages. It concludes that while visual AI testing has advantages, a hybrid approach combining it with traditional testing may be best to avoid false positives.

The document discusses using specification-by-example (SBE) techniques to build products right. It highlights that SBE can help ensure everyone understands requirements, provide precise specifications to eliminate ambiguities, and create efficient acceptance criteria. It also notes that SBE specifies business goals, derives scope through examples, and builds living documentation through refining specifications and automated validation.

The acceptance criteria provides the majority of what needs to be tested from the user story. To achieve maximum test coverage, time should be taken to consider the workflow, boundaries, and negative scenarios based on the acceptance criteria. Writing test cases in this way can help identify requirement gaps, prevent code re-work, and save the entire team from unnecessary headaches by ensuring the test cases remain up-to-date and address all aspects of the user story.

The document discusses test automation architecture and provides recommendations for building an effective architecture. It recommends prioritizing unit testing and API/service layer testing over GUI testing to create a testing pyramid. Unit tests should be isolated and test small pieces of code, while API tests can test application logic through service calls. GUI tests should be limited in number and used to test broad end-to-end scenarios, not every small scenario. The goal is to have fewer, more stable automated tests rather than many fragile tests. Following best practices like testing different layers, prioritizing types of tests, and continuous refactoring can help create a maintainable and effective test automation architecture.

Software teams are faced with the prospect of building a product, only to have unexpected shifts in customer demand, changes in the competitive landscape, or swings in the economic climate undermine their plans and turn their product into expensive waste. What is an entrepreneurially-minded software developer, designer, or tester to do? Thomas Vaniotis guides you toward a shift in thinking that is manifest in the Lean Startup and Lean UX movement. Learn to value information that comes from quickly exposing an idea to market pressures rather than considering the delivery of a particular feature as the goal. Identify wasteful activities in your product cycle and re-invest that energy by innovating around the build-measure-learn loop that drives value. Lean thinking and using meaningful production data to drive decisions will assist you—whether a tester, developer, product manager, or designer—in operating under the uncertain conditions of modern markets, regardless of your company’s age or size.

Many organizations are actively searching for the perfect product owner—a unicorn who knows all about the product, anticipates the market, innovates, and improves the product’s quality and architecture, all while making and meeting commitments to the organization. That's a difficult if not impossible role to fill. So, how can we achieve these goals? Various enterprise patterns of scaling the product owner are emerging including the technical product owner, the proxy, the product owner team, the program team, the market manager, and the innovator. Tim Wise describes where each is applicable in large enterprises. Learn how to apply these approaches to both a service organization and a product development organization. Get a glimpse into the evolution that will influence product owner roles in large organizations as companies scale agile in the enterprise. Leave with knowledge of patterns that are emerging in large enterprises around the product owner.

Never underestimate the power of sharing the testing team’s achievements, lessons learned, challenges faced, roadblocks encountered, and the enriching solutions found. Jyothi Rangaiah says as testers we must be ready to nurture the needs of testing and testers in the organizations we serve. Only people, learning every day and questioning the norm, can and will move testing forward. Getting into this learning mode requires awareness of the need to improve. Jyothi discusses the importance of sharing the testing team’s everyday challenges and achievements with all involved. One person who cares for the quality of testing often can change the perception of testing in an organization. The internal corporate social networking site can be a good start to spread awareness and knowledge about testing. To test better, encourage and uphold the ways that are already beneficial. Network with the interested and the influential, and help each other understand the current state of affairs of testing.

Adopting agile development methods in a conservative environment can be a daunting and time-consuming venture, facing resistance at all levels of the organization. You may wonder: Will this organization ever get with the times? Will our leaders ever change their way of thinking? Brian Duncan shares personal experiences and lessons learned in bringing an agile development mindset to two distinct organizations—a bottom-line product-driven software development organization, and the conservative, risk-averse Space Department at the Johns Hopkins Applied Physics Laboratory. Sharing the Good (what worked well), the Bad (what set us back), and the Ugly (what we had to abandon), Brian shows how to bring a slow-to-change organization into the forward-thinking agile methods of today. He presents practical approaches (adoption committees, grassroots techniques) and creative endeavors (free classes, an innovation lab) along with their impact on the organization. With persistence and a multifaceted approach, even risk-averse organizations can adopt agile.

Today, organizations are rapidly deploying mobile versions of their customer-facing and internal applications. With the prevalence of more agile-based approaches and the challenge of an ever-increasing diversity of devices and OS versions, testers are being asked to accomplish more testing in less time. Rachel Obstler shares how leading enterprises are improving the efficiency of their mobile testing using automation, and how they identify the right processes and tools for the job. Sharing some fascinating statistics from their recent mobile quality survey of more than 69,000 mobile app developers and QA organizations in the top US enterprises, Rachel dives into the challenges identified in the survey and shares five clear ways to improve your testing process: implementing a collaborative agile process, optimizing with a development tool that naturally facilitates testing, using a combination of real and emulated devices—and when to use them, and more.

Know any testers who have bugs opened more than a year ago and still sitting in their defect queue? More than two years ago? Three? The fact is that many software development efforts are focused on delivering new features and functionality, leaving workarounds in place for bugs released in prior versions of applications. Often these defects seem relatively minor—we all have some workarounds for customers—but these are still bugs and ultimately should be dealt with. If you are seeking effective methods to close out those bugs once and for all, Shaun Bradshaw shares his experience eradicating aging bugs—in a Bugfest! Shaun shows how to effectively use kanban techniques to bring visibility to a myriad of outstanding problems left over from previous releases as well as to order and prioritize the work to clear out the nastiest, most offensive defects—and ultimately exterminate those pesky bugs!

In agile projects, when the cycle from ideas to production shortens from months to hours, each software development activity—including testing—is impacted. Reaching this level of agility in testing requires massive automation. But test execution is only one side of the coin. How do we design and maintain tests at the required speed and scale? Testing should start very early in the development process and be used as acceptance criteria by the project stakeholders. Early test design, using a business domain language to write tests, is an efficient solution to support rapid iterations and helps align the team on the definition of done. These principles are the basis of acceptance test-driven development practices. Laurent Py shows you how the use of business domain languages enables test refactoring and accelerates automation. Come and learn how you can leverage acceptance tests and key test refactoring techniques.

Load testing is often one of the most difficult testing efforts to set-up—in both time for the deployment and cost for the additional hardware needed. Using cloud-based software, you can transform this most difficult task to one of the easiest. Charles Sterling explains how load testing fits into the relatively new practice of DevOps. Then, by re-using the tests created in the load testing effort to monitor applications, the test team can help solve the challenges in measuring, monitoring, and diagnosing applications―not just in development and test but also into production. Chuck demonstrates web performance test creation, locally run load test creation, cloud executed load test to the cloud, application performance monitoring (APM), global system monitoring (GSM), and usage monitoring (UM) for near real-time customer input for your application.

This is the story about the Microsoft Developer Division and their two-year journey to agile—from shipping every three years to shipping every three weeks. In the old days, long stabilization phases were part of its DNA. Managers were rewarded for micromanagement. Commitments were made months in advance. Maintaining the appearance of meeting commitments was valued over transparency. Gregg Boer shares how this organization within Microsoft transitioned to one that values agile principles—controlling technical debt, enabling teams, eliminating bogus commitments, and rewarding transparency. When applying agile to such a large, traditional organization, the key to success is allowing autonomy at the team level, while ensuring alignment with the organization. Gregg shares successes as well as colossal failures. Learn how management sets direction while teams own their own backlog, how communication up and down can be transparent and healthy, and other lessons on their journey to agile.

DevOps is gaining popularity as a way to quickly and successfully deploy new software. With all the emphasis on deployment, software quality can sometimes be overlooked. In order to understand how DevOps and software testing mesh, Glenn Buckholz demonstrates a fully implemented continuous integration/continuous delivery (CI/CD) stack. After describing the internals of how CI/CD works, Glenn identifies the touch points in the stack that are important for testing organizations. With the now accelerated ability to deliver software, the testing groups need to know how this technology works and what to do with it because swarms of manual testers will not be able to keep up. Glenn demonstrates where and how to use automated testing, how to collect and make sense of the massive amount of test results that can be generated from CI/CD, and how to usefully apply manual testing.

Software processes are commonly portrayed using machine metaphors in which consistency is highly prized. Frequently, organizations set up Centers of Excellence in a well-intentioned effort to create enterprise consistency. Steve Adolph reminds us that, in reality, software development takes place in a diverse ecosystem of corporate policies, competing interests, personal agendas, personality types, and a variety of formal and informal relationships. An aggressive top down imposition of practices is like sending a bulldozer through an ecosystem. This can create a prized consistency, but it also can destroy the environment’s productive vibrancy. It does not matter if the bulldozer says waterfall or agile on the side—it’s still a bulldozer. How do we live in harmony with our ecosystem? We can start by replacing machine metaphors with biological ones about leveraging and embracing diversity. Then use these metaphors to interpret two case studies of how organizations either bulldozed their ecosystem or learned to boost their productivity by living in harmony with it.

All too often, companies adopt a mission to “go agile” before truly understanding what that entails. Business managers are quick to jump on the agile bandwagon, believing that going agile will magically make projects happen faster. Teams are getting certified in Scrum believing it is the silver bullet that will suddenly make everyone more productive. Inevitably, cracks begin to develop, and expectations are not met, leaving everyone questioning the value of going agile at all. Heather Fleming and Justin Riservato say there is a better way! The truth is that going agile will result in more productive teams and faster delivery of projects—but only if everyone can agree on the rules of the game. Learn why gaining consensus on the principles of agile is more important than implementing a specific process. Explore how having three key conversations—about saying no to deadlines, ensuring business partner engagement, and experimenting with process—up front can save you from an agile disaster.

As cloud computing becomes of strategic importance in the enterprise, part of the solution is no longer on-premise but in the cloud, adding a layer of complexity. Edwin Chan demystifies performance testing of cloud systems and applications by addressing the following key questions: Is performance testing of cloud systems fundamentally different from testing on-premises applications? What are the best practices for performance testing of both cloud and on-premises systems? Performance testing of cloud systems is essentially the same as that of its on-premises counterpart with the exception of the key consideration of network latency. After clearing common misconceptions, Edwin shares the hot topic best practices—adopting an agile/lean methodology, conducting early performance testing, and automating the injection of test data. Discuss the challenges the testing team faces in these days of disruptive and fast-paced technology changes. Take back and apply some of the best practices that fit your organization’s need.

As testers, we can feel overwhelmed by the sheer volume of things that require our attention. We are pressured to meet the demands of a fast-paced development environment while grappling with the extreme complexities inherent in today’s software. How can we remember everything while prioritizing our work in a way that allows us to test thoroughly and with confidence? Kirk Lee shares how the proper use of checklists provides a lightweight yet powerful solution. Kirk explains how checklists can prevent forgetfulness, assist in assessing what is really important, and most importantly, free our minds from the mundane so we can reach the deeper levels of thought required to find the nastiest bugs. Take back foundational checklists and learn how to adapt them to your specific circumstance through defect and root-cause analysis. Kirk shares checklists focusing on quality attributes, test types, heuristics; and functional, security, performance, automation, and mobile testing.

As applications for smartphones and tablets become incredibly popular, organizations encounter increasing pressure to quickly and successfully deliver testing for these devices. When faced with a mobile testing project, many testers find it tempting to apply the same methods and techniques used for desktop applications. Although some of these concepts transfer directly, testing mobile applications presents its own special challenges. Max Saperstone says if you follow the same practices and techniques as you have before, you will miss critical defects. Learn how to effectively test mobile applications, and how to add more structure and organization to generate effective test ideas to exploit the capabilities and weaknesses of mobile devices. Max shares first-hand experiences with testing mobile applications and discusses how to address various challenges. Work on real problems on your own device and learn firsthand how to be productive while testing mobile applications.

Warning! You can still build the wrong product using agile. In Eric Ries’ book The Lean Startup, he poses the question: What if we found ourselves building something that nobody wanted? In that case, what would it matter if we did it on time and on budget? We often assume the Product Owner is smart enough to define the right product. But what if we are wrong? Michael Hall shares lean startup principles and how they can be applied to ensure that the product we are building is righteous. Learn new agile concepts such as hypothesis-driven project vision, knowledge broker personas, learning maps, minimum learning product, experiment backlogs, experiment test iterations, validated learning, and pivot/persevere decisions. Case studies and Michael’s first-hand product experience emphasize the learning points. New and mature agilistas alike will leave the session armed with Lean Startup agile techniques that can be applied immediately on their agile projects.