| Concept | Definition / Purpose | Example |
|---|---|---|
| Test Reporting | Communicates test status and quality insights to enable stakeholder decisions. | Report shows only 65% of critical product risks fully tested, triggering more focused testing. |
| Tailored Reporting | Adjusting report focus based on test level and stakeholder needs. | Business gets risk coverage metrics; Dev gets defect breakdown and code coverage. |
| Structural Metrics | Metrics such as code coverage – used mainly at component and integration levels. | Statement coverage = 95%; used by developers to check test completeness. |
| Risk-Based Metrics | Track how well test cases mitigate identified product risks. | 90% of high-risk scenarios passed; 10% failed → focus attention on remaining risks. |
| Defect Metrics | Analyze trends, status, and root causes of defects. | 60% defects from missed requirements → improves test basis for next cycle. |
| Test Progress Metrics | Track planning vs. execution status, test case outcomes, and effort. | 80% of planned tests executed; 10% blocked due to environment. |
| Coverage Metrics | Indicate test completeness relative to requirements, risk, or code. | 85% requirement coverage, 70% code coverage → insights into residual risk. |
| Cost/Effort Metrics | Compare planned vs. actual costs and resource utilization. | Planned: 300 hrs, Actual: 420 hrs → cost overrun. |
| Trend Correlation | Combine metrics to reveal deeper insights and support key decisions. | Correlate open defects vs. executed tests to decide if testing can stop. |
Q1 (K4 - Long Scenario):
You are the Test Manager for a payment gateway project. You collect these metrics:
-
95% test cases executed
-
40 high severity defects reported, 32 fixed
-
90% requirement coverage
-
15% of identified high-risk areas show failed test results
What should your report emphasize for stakeholder decision-making?
A. Focus on 95% execution and recommend release
B. Recommend release since requirement coverage is high
C. Highlight failed tests in high-risk areas and recommend delay
D. Only share defect fix rate
✅ Answer: C
🧠 Explanation: Executed ≠ Passed; failed tests in high-risk areas mean potential business failure.
Q2 (K4 - Short):
Which combination of metrics best helps stakeholders assess test effectiveness?
A. Number of open defects + number of testers
B. % requirements covered + % risks mitigated + defect severity trend
C. Code coverage + cost metrics only
D. Test execution count + lines of code
✅ Answer: B
Q3 (K4 - Long Scenario):
You're reporting for a mobile banking release. Code coverage is 100%, but 10% of critical use cases failed in UAT. What do you report?
A. Recommend release – code coverage is high
B. Suggest fixing only UAT bugs
C. Emphasize risk of releasing with failed business scenarios
D. Recommend retesting component tests only
✅ Answer: C
🧠 Explanation: Structural coverage ≠ risk mitigation. Business use cases are what matter here.
Q4 (K4 - Short):
Your test report shows an increasing number of open defects and a plateau in test execution. What does this trend suggest?
A. Environment is stable
B. Testing is complete
C. Test team is inefficient
D. Potential release delay or risk; analyze defect density and blocking tests
✅ Answer: D
Q5 (K4 - Short):
Which metric pairing is most useful to determine release readiness?
A. % test cases skipped + test effort
B. Defect severity trend + risk coverage
C. Code coverage + number of testers
D. % blocked tests + test cost
✅ Answer: B
1. Scenario – Product Risk Coverage & Decision
You are preparing a test closure report. The data shows:
-
85% requirement coverage
-
65% of product risks tested
-
100% high-severity defects fixed
-
90% test cases passed
What is your most critical observation to report to stakeholders?
A. Requirement coverage is good
B. Execution pass rate is acceptable
C. Product risk coverage is low → recommend additional testing
D. High-severity defect fix = safe to release
✅ Answer: C
2. Scenario – Trend Correlation
Your team presents a graph showing the number of test cases executed vs. open defects over time. The curve flattens for test execution, but defects keep increasing.
What does this indicate?
A. Test cases are redundant
B. Need more testers
C. Defects may outweigh benefits of further execution → consider halting
D. Release is safe
✅ Answer: C
3. Scenario – Stakeholder-Specific Reporting
Marketing stakeholders want to know if user journey risks are mitigated. What should you show them?
A. Code coverage report
B. Product risk coverage, especially on user-facing features
C. Unit test defect density
D. Number of retests done
✅ Answer: B
4. Scenario – Cost Overrun Evaluation
Planned test effort: 300 hours; Actual: 500 hours. Major cause: test environment downtime.
What should your report include?
A. Blame DevOps
B. Track defect trends
C. Include root cause, cost deviation, and mitigation plans
D. Recommend test case reduction
✅ Answer: C
5. Scenario – Early Stop Decision
Test execution trends show fewer defects being found over 5 consecutive days, and all critical defects have been fixed.
Exit criteria: 90% test pass and no critical open defects.
What is the correct action?
A. Stop testing and release
B. Re-test everything
C. Ignore trends
D. Keep testing until all test cases are executed
✅ Answer: A
6. Scenario – Incomplete Testing Insight
Report shows 100% execution but only 70% requirement coverage.
What’s the issue?
A. Testers are inefficient
B. Test planning missed requirements
C. All is good
D. Retest all
✅ Answer: B
7. Scenario – Defect Trend Analysis
70% of defects originate from requirements misunderstanding.
Which action should be recommended in your report?
A. Add more testers
B. Improve requirement review process
C. Stop testing until better requirements
D. Increase code coverage
✅ Answer: B
8. Scenario – Regression Quality
Regression execution is complete. 30% of tests failed, mostly in modules with high change frequency.
Which metric justifies prioritizing those modules?
A. Code coverage
B. Defect severity
C. Change-based test coverage + historical defect density
D. Number of testers
✅ Answer: C
9. Scenario – Metrics Overload
A stakeholder asks for only essential metrics to decide on release.
Which should you choose?
A. % test passed, % risk coverage, severity-based defect trend
B. Number of test cases written
C. Code comments
D. Number of testers working
✅ Answer: A
10. Scenario – Tailored Report Fails
You present a highly technical test report to the finance department. They express confusion and disinterest. What went wrong?
A. Wrong data
B. Stakeholder-specific tailoring was missing
C. Not enough graphs
D. Cost wasn’t included
✅ Answer: B
No comments:
Post a Comment