Dealing with large data set and complexity in your testing

  • Published on
    23-Feb-2016

  • View
    44

  • Download
    0

DESCRIPTION

Dealing with large data set and complexity in your testing. Jae-Jin Lee . Search results (Facts). Google/Bing Possible number of inputs is close to infinity There are huge amount of data source Algorithms (placement) are very complex Expedia - PowerPoint PPT Presentation

Transcript

Dealing with large data set and complexity in your testing

Dealing with large data set and complexity in your testingJae-Jin Lee 1Search results (Facts)Google/BingPossible number of inputs is close to infinityThere are huge amount of data sourceAlgorithms (placement) are very complexExpediaPossible inputs are not as huge as Google, but the same input can return different results based on dates, traveler info and other factors.There are huge amount of inventoriesAlgorithms are very complex and direct impact to the business2Search results (Facts)Google/BingPossible number of inputs is close to infinityThere are huge amount of data sourceAlgorithms (placement) are very complexExpediaPossible inputs are not as huge as Google, but the same input can return different results based on dates, traveler info and other factors.There are huge amount of data source(world, inventories)Algorithms are very complex and direct impact to the business3Testing Challenges Test input selectionData is not organized in a way to be testedEquivalent partitioning is hardRandomness? Coverage?Verifying mechanismHow do we get expected result?RE-implement the algorithm?474,000,000 results for "Seattle search

4Good newsAlgorithms are complex but definedWe have a full access to data sourceHistorical data/statistics are availableNot all the results are equally important

5Risk analysis / assessment6Risk analysis / assessmentQuestion the project (Is it feasible to do it?)Practical risk analysis Understand the risk on business perspectiveUnderstand the likelihood of faults from development perspectiveValidations to be doneTest casesCome up with list and reviewed by entire project team

7Risk analysis / assessmentQuestion the project (Is it feasible to do it?)Practical risk analysis Understand the risk from business perspectiveUnderstand the likelihood of faults from development perspectiveTest casesValidations to be doneCome up with list and reviewed by entire project team

8Risk analysis / assessmentQuestion the project (Is it feasible to do it?)Practical risk analysis Understand the risk from business perspectiveUnderstand the likelihood of faults from development perspectiveTest casesValidations to be done Summarized it and reviewed by entire project team

9Understand data source Data source is trusted source of test case validation / verification mechanismModifying data source should be piece of cakeInsert, delete, update rows or execute sprocsSetup and tear downNo assumption on data source

10Test input selectionHistoric data and statisticsPriority from risk analysisCreativity and product knowledge to breakRadom valid inputs from bucketing (do as much as you can and log the useful details)Hard-coded data11Decompose the algorithmExercise each logic separately by controlling data source and dependenciesWorking with dev for testability or hooks (architecture, logs, and etc.)If possible, implement algorithms for happy path in your test automation

12Heuristic approach helpsIs there a place where good enough result acceptable?Seatttle (three ts)Is that in the list?Is that in the first 10 results? 13Hybrid approach (manual + automation)Integration environmentCombine humans intuition/product knowledge and machines powerful diligenceExecute manually and validate using test validation code (turning on logs)Requires decoupled class design in your automationUI(JavaScript) broke the functionality

14Your thought?15

Recommended

View more >