Brilian S.
Real work, real clients, but the payment model puts all the risk on the reviewer
I completed close to 60 scientific paper reviews on a rubric-auditing project. I was paid for 2. The core issue is that acceptance criteria were never disclosed during onboarding. We were told this was to prevent gaming of the metric, which I understand in principle — but the practical effect is that you complete dozens of hours of work with no way to know the standard you're being measured against and no feedback loop to correct course. Submissions were filtered by an automated screening system. When I asked why so many of my reviews were rejected, I was told the system flags behaviors like submitting shortly after opening a task or pasting content into the tool rather than typing it in. That explanation came after the fact. If those are payment-eligibility conditions, they belong in onboarding documentation, not in a message sent once the project is already over. Reviewers who prepare their analysis carefully offline — which is normal practice for serious paper review — have no way of knowing they are disqualifying themselves. The automated auditor itself was unreliable. Reviewers on the project repeatedly encountered auditor flags that were factually wrong about the papers, and the tooling did not allow mischaracterized rubric items to be corrected — only reweighted. This was raised internally and acknowledged. Rejections from that system came with no appeal route. Of my ~60 reviews, 9 were ever surfaced to me: 2 accepted, 7 returned for revision. I was never able to submit those 7 revisions. The project went silent for roughly two months, then was terminated. Work that the reviewers themselves judged worth fixing was simply discarded. Pros: genuine frontier-lab work, intellectually substantial, well-funded company, responsive when issues are raised in channel. Cons: undisclosed acceptance criteria, unreliable automated rejection with no appeal, no feedback on rejected work, abrupt project termination with pending revisions stranded, and a compensation model where you carry the full cost of work that may never be paid. If you take on work here, assume anything you produce may go uncompensated and price your time accordingly.