How We Evaluate

Our testing standard

Before any tool earns a recommendation here, it has to clear two bars:

1. Real production use, minimum 30 days. We run the tool on a live product, paying real money where applicable, for at least a month before recommending it. We do not review tools we have only trialled, watched a demo of, or read about. Recommendations come from operating the tool against actual users and traffic - not a sandbox.
2. A specific failure mode tested for. Every tool gets pushed against the thing it is supposed to handle. For analytics, we deliberately polluted our test traffic to see how each platform handled the noise. For payments, we processed real transactions through each candidate before naming a winner. For email, we sent volume to a real list and watched the deliverability numbers.

Tools that fail either bar do not get recommended, even where we have an affiliate relationship. The article will say so honestly.

What we run, what we ship on

PennyBlack is part of Raging Orangutan Holdings - the same operation behind BeatsVine, MainMenu, OrangutanFamily, and other live products. The tools we recommend here are mostly the tools running those sites.

That is the credibility model. Recommendations don't come from a review lab - they come from the stack we operate. The What We Use page lists the current production stack with real costs and the alternatives we considered. Where we have migrated away from a tool, the relevant article says what we replaced it with and why.

What we look at

For each tool category, the criteria that actually matter to solo founders and small teams:

• Pricing transparency - what does it actually cost, including the bits that aren't in the marketing example? Hidden fees, usage tiers, overage billing.
• Time to first value - can one person get this running in under an hour without escalating to a sales call?
• Data ownership - can you export your data when you want to leave? Are you locked into proprietary formats?
• Privacy and compliance - does it respect user privacy by default? Ready for GDPR-equivalent regulations without bolt-on consent banners?
• Support quality - when something breaks, who replies, and how fast?
• Failure modes - what happens when this tool fails? Does it fail loudly (good) or silently (bad)?
• Self-hosted alternative - if a viable open-source equivalent exists, what is it, and what is the real maintenance burden?

What we do not do

• No numerical scores or star ratings. A tool is either the right choice for a specific use case or it is not. Ranking analytics tools out of 10 is a fake exercise that hides the reasoning behind the number.
• No paid placements. Commission rates play zero role in rankings. Where we have an affiliate link to a tool we don't actually recommend, we don't link to it.
• No reviews of tools we have not run. If we have not shipped with it, we say so. If a tool is on our roadmap to test but we haven't yet, we don't fake having opinions about it.
• No "balanced" hedging. Every comparison names a winner for a specific use case and what we would skip. Articles that try to make every tool sound equally good are the affiliate-site default we are explicitly avoiding.
• No vendor demo accounts. We pay for what we test where there's a free tier we use that, but we don't accept comped or extended trials from tool providers - they skew the experience.

The self-hosted question

Every comparison on this site asks: "what if I want to own this completely?" Where a viable self-hosted alternative exists, we include it as a first-class option - not a footnote. We run self-hosted tools on our own infrastructure (Umami for analytics, Uptime Kuma for monitoring, others) and can speak to the real maintenance burden, including the cases where the self-hosted route turned out to be more work than it was worth.

Staying current

SaaS tools change frequently. Pricing changes. Features ship. Acquisitions happen. We revisit articles when something material changes, when reader feedback indicates an article is out of date, or when our own use of a tool reveals something we didn't catch the first time. Every article shows when it was last updated.

Last updated: 8 May 2026