How We Test Software

This page sets out Impulsec’s standards for product testing. The assessment described in an individual article determines what was actually tested; this methodology does not imply that every product or feature in the archive has undergone every check below.

Direct use and research are different

Hands-on testing means a named tester directly used the software. Reading documentation, viewing vendor screenshots or collecting customer feedback is research. An interactive demonstration may support observations about that demonstration, but does not establish how a full installation behaves.

Where access is limited, an article must keep its conclusions within that limitation.

What a testing record should contain

A direct-use assessment should record the tester, assessment date, product and plan, relevant app version, device and operating system, tasks attempted, observations and limitations. Screenshots or other records should support material findings without exposing personal information.

An article’s testing summary should explain enough of these conditions for the reader to understand the findings. Results from one platform must not be presented as results from another.

Setup and everyday use

Depending on the product, relevant tasks include installation, account creation, permissions, configuration, navigation, key feature use, cancellation and removal. An assessment should report failures and restrictions as well as successful tasks.

Support responsiveness is a test finding only when the tester actually contacted support. Reading a help centre can support a judgement about documentation, not a claim about response time.

Parental control and family safety

Relevant checks include device enrolment, screen-time rules, app and website restrictions, supported alerts, location features and parent-account controls. Android and iPhone findings must be reported separately where capabilities differ.

Tests should use controlled scenarios and minimise collection of personal data. A feature working in a test does not prove that it can prevent harm or identify every risky interaction. The assessment must consider proportionality, privacy and less intrusive alternatives.

Antivirus and device protection

Direct use can assess installation, settings, scan operation, usability and observed system impact. Malware-detection claims require suitable independent testing or a properly documented specialist test. A benign test-file check is not a comprehensive malware-protection result.

Password managers

Relevant tasks include account setup, importing test records, saving and filling logins, supported passkeys, sharing, recovery and account protection. Security-architecture claims require technical documentation or other appropriate evidence beyond routine use.

VPNs and privacy tools

Relevant checks include connection setup, supported protocols, stability, leak behaviour and kill-switch behaviour under stated conditions. Performance comparisons need a stated baseline, location, connection and measurement method. A successful leak check does not prove a provider’s logging practices or complete anonymity.

Identity protection

Assessment should distinguish alerts and monitoring from prevention, restoration support and any insurance benefit. Subscription access alone cannot establish the effectiveness of recovery assistance or the outcome of an insurance claim. Country eligibility, exclusions and service terms matter.

External evidence and limitations

Ownership, privacy policies, audit reports, published security research and subscription terms require separate source checks. Vendor assertions should be attributed when independently confirming them is not possible.

Testing findings apply to the version, conditions and time assessed. Later changes may require retesting. Ranking decisions are explained in our Review Methodology. Concerns can be reported through our Contact page.