How to choose a QA vendor for a security product
Shortlist two or three vendors from published evidence before the RFP, using the same eight dimensions the ranking scores.
Research desk, cybersecurityqavendors.com. Published 2026-11-11.
Short answer
Start from your product class, not from vendor size. Keep vendors whose published cases name your class and the test types your release needs, drop any whose not-ideal-for line matches your situation, then check certificates and engagement length before you send the RFP.
What to decide before you look at vendors
Write down three things: the product class, the test types the next two releases need, and the platforms you support. An EDR1 agent needs compatibility and performance work across operating system builds; see the endpoint facet. An IAM2 product needs protocol and permission testing. A SIEM3 needs detection regression on real log formats.
These three answers decide which facet page to start from and which dimension scores matter most to you.
The shortlist procedure
- Open the facet for your product class. The facet table ranks vendors on that class's evidence only, so a vendor with many cases in another class does not rise above one with a case in yours.
- Read each candidate's not-ideal-for lines first. Each line names a situation the evidence does not cover, with a source. Drop a vendor when a line describes your situation.
- Check dimension 3 against your test types. Test-type depth counts functional, automation, API, performance, compatibility and detection regression work evidenced on security products. For an agent product, a vendor with no evidenced performance work has shown no case of measuring resource cost on a host.
- Check dimension 4 for engagement length. A security product changes with every OS release; a long engagement with a security client is evidence that the vendor has tested through several of those releases, not proof of quality.
- Check dimension 6 before legal review. Your security team will ask for an ISO 27001 certificate number and a SOC 2 report; the vendor page shows whether either is published with a reference.
- Send the RFP to two or three vendors. Use the questions in the RFP guide and ask each vendor to answer with a case, a reference or a document.
Signals that do not count
Logos without a case, award badges, listicle placements and star ratings are not evidence of security-product QA. The methodology excludes all of them.
Penetration testing for a client that is not a security product company is a different service. A vendor that lists it as security experience has shown nothing about testing your agent or detection pipeline.
When the ranking has no vendor for your class
Several product classes have fewer than three ranked vendors with evidence, so no facet page exists for them. The list is on not published. Suggested practice in that case: weight dimensions 3 and 4 higher and ask each vendor for a reference in an adjacent class.
- 1 EDR: Software that watches laptops, servers and phones for suspicious behavior, records what it sees, and alerts administrators so they can investigate and respond. Source
- 2 IAM: The practice of managing user identities in an organization and deciding which roles and access privileges each person or account gets. Source
- 3 SIEM: A system that collects log data from many sources into one place so security teams can search it, correlate events and spot attacks. Source
Questions buyers ask
What should a security product company look for in an outsourced QA partner?
Published QA cases on your product class, the test types your release needs, an engagement over 12 months with another security vendor, and certificates with reference numbers. These map to dimensions 1 to 4 and 6 of the ranking.
How do I verify a QA vendor has real security product experience?
Open the case study and check that it names the product type and describes testing, not development or penetration testing. Then ask the vendor for a reference call with that client or a similar one.
When in the development cycle should a QA partner start?
When the test hosts, OS images and build access are ready, and before the release that needs them. Agent products need the compatibility matrix set up before the first test cycle.
How should a QA vendor protect our code and test data?
In writing: where builds are stored, who can access them, how test data is kept free of customer content, and how access ends at contract exit. Ask for the ISO 27001 certificate number and check it with the issuing certification body.
Can we interview the QA engineers before signing?
Ask for it in the RFP. Interview the people who will run your test environment, not only the account lead, and ask each one about a security product they tested.
Who owns the test suites after the engagement ends?
Set it in the contract: the automation code, test data and documentation live in your repository from day one. If suites live in the vendor's systems, switching vendors requires an export first.
What are the main risks of outsourcing security product QA?
Builds and detection content leaving your control, a team that knows web apps but not agents or detection pipelines, and test suites you cannot take back. Each risk has a matching RFP question in the companion guide.