This is a tricky question. Of course, if OSHWA has to evaluate every request for certification that would, we hope, take a great deal of effort due to many projects opening up in the future! Thinking about fees might well be needed.
However, the risk of self-certification is that some projects claim the certification without meeting the requirements, either through malice or ignorance, and if such a project attracted media attention it could significantly dilute or damage the OSHWA/open hardware brand and concept.
Depending on the community response on other questions it might be worth looking at a hybrid model. Let us assume a spectrum of certification is available. Lower tiers, say “bronze,” might be available for self-certification, but the ‘gold’ tier might require formal submission and evaluation by OSHWA. This top tier process may or may not require fees, and may require projects to supply a portfolio of supporting evidence (to cut down OSHWA effort!). Projects failing to submit appropriate documentation could have their applications rejected with minimal effort.
The pre-approval process which enables ‘gold standards’ to be maintained could also be something limited, say, to projects which were looking at serious scale (whether commercial or otherwise). This would mean small projects could still get self-certified to a high standard of open hardware-ness, but if they were deploying some larger number of units from a central source then the pre-approval extra-strong certification would be strongly recommended, as this would help such projects demonstrate their seriousness in the market/community. This would cut the risk of a large scale project maliciously self-certifying and then bringing the open hardware movement into disrepute, as any large scale project would be required to be formally checked if it wanted to use the certification.
For comparison, I would point at the work of the Open Definition Advisory Council. This group evaluate licences which wish to be considered as compliant with the Open Definition (and also are stewards of the Open Definition itself). Because licence proliferation is a bad thing, and licences are designed for reuse, this should be a finite job; there are not an ever-increasing list of new licences to evaluate and never will be. Openness can grow and thrive without more licences being approved, so the manual process is OK. It also adds value as there are subtleties around language and interpretation of both licences and Open Definition at times where discussion and consultation is very valuable. The Open Definition is carefully policed, therefore, with every licence being checked before approval, and as a ‘gold standard’ and a binary open-or-not, this is important for the future of open.
However, OSHWA faces a very different situation. We wish to see more open hardware, and therefore more products which are certified (if certification is chosen). The number of projects/products requiring certification is something we want to see grow. We also hope that compliance is something reasonably clear and not subject to debate about language or subjective argument. So a manual process of evaluation is not so good - except, perhaps, for a small subset of potential open hardware projects, where the effort required adds value (and ideally can be compensated in some way).