We have refined a previously developed exposure prevention rating method and adapted it to small business manufacturing settings. This rating method complements quantitative exposure assessment with a systematic and efficient assessment of prevention and control efforts with an emphasis on upstream prevention and control. It has been designed for use by researchers and evaluators as an intervention process and effectiveness evaluation tool . It could also be used by practicing OHS professionals with limited budgets, which is of particular value in small business settings where OH&S resources are often very limited.
Field application of the SBEI rating method in 25 small manufacturing worksites has shown it to be capable of providing common metrics across various hazardous substance exposures found in 149 manufacturing processes or areas. Broad applicability, good discriminatory power, and excellent inter-rater reliability have been demonstrated. The main improvements over the previous (initial) version  were:
Changing from two-level 'Major' and 'Minor' indicators to three levels (Major/Moderate/Minor), as detailed in Tables 2, 3, 4, 5, &6;
Revisions to the scoring algorithm so as to better account for concurrent potential or protective factors (e.g., only rating a carcinogen as a concern if used on a daily basis in substantial amounts);
Substantial revisions to the Material Potential assessment, in particular improvements in accounting for multiple substances being used in a single production process or area;
A substantially expanded assessment of inter-rater reliability; and
Full explication of scoring methods and inclusion of the actual Checklist and Guide.
Further development work is still needed. Most importantly, this includes validation of SBEI scores against quantitative and other exposure metrics.
The data gathered might be improved by additional interview of the site staff person guiding the walk-through assessment, line supervisors, and workers in the area. While such interviews would surely provide a deeper and broader assessment, we anticipated that this would not be feasible in most study sites due to a combination of production pressures, the sensitivity of OHS issues in many workplaces, and other concerns. Other concerns include the challenge of reliably interviewing workers in private while being guided by someone who is usually a management representative (in order to get frank responses and data of comparable quality across all areas assessed), and how to combine data in situations where different interviewees respond differently to the same or similar questions. In summary, we believe that incorporating interviews of walk-through guide, line supervisors, and workers would overly complicate the administration of the SBEI checklist. This would also represent a different measure–one based on site employee perceptions rather than an outside industrial hygienist. Our strategy in the Healthy Directions-Small Business study as a whole has been to gather data on worker perspectives (through confidential surveys at individual worker level) and OHS programs (organizational level) separately and in parallel to the SBEI assessments (physical environment level) . Taken together, these three levels provide a comprehensive assessment of OHS conditions for both needs assessment and evaluation purposes. Cross-comparison of OHS performance across these levels and measures could be the subject of further analyses of these data (e.g., to demonstrate convergent validity). Separately, these three levels each provide a different way of understanding OHS hazards or exposures. The SBEI assessment discussed in this paper provides a low-cost, feasible method by which to independently evaluate hazardous substance exposures and prioritize interventions.
The distributions of ratings showed reasonable discriminatory power of the SBEI exposure prevention rating method, with a general pattern towards low Potential and high Protection ratings, and a distribution of overall SBEI scores that was strongly skewed towards the favourable end. A similar pattern was observed previously in large manufacturing worksites in the Wellworks-2 trial . The frequency of favourable ratings in our sample may be artefactually elevated relative to the full population of manufacturing worksites due to the selection biases inherent in this study. Participating companies had to voluntarily agree to occupational health intervention together with health promotion if they were randomized to the integrated intervention group . Thus companies that have exposure concerns or that do not place a high priority on occupational health would have been less likely to participate. We would also note, however, that many companies that expressed willingness to participate noted the OHS consultations as an important incentive.
Despite the likely overestimate of favourable ratings in comparison to the full population of small manufacturing businesses, a gradient of intervention needs was identified in our sample. Significant fractions of the sample received the very poor (~10% with score of 5 or 6) or intermediate (~32% with score of 3 or 4) SBEI scores. A strength of these scores is that each has corresponding intervention recommendations to guide the user in shifting prevention and control efforts upstream. In this regard, the detailed Potential/Protection matrix and SBEI scores perform a detailed needs assessment and prioritization function as well as providing baseline measures for effectiveness evaluation. As noted in the Introduction section above and elsewhere , occupational disease constitutes a substantial and inequitably distributed burden. Thus prioritising of intervention using the SBEI approach, which in this sample identified 10% in need or urgent attention, represents an expeditious and targeted means to reduce this burden.
Inter-rater reliability of the 5 Potential and Protection ratings used to compute SBEI ratings was good to excellent, and the overall SBEI exposure prevention summary scores demonstrated the best reliability of all. Because the two observers were both involved in instrument and protocol development, however, this may overestimate the inter-rater reliability that would be observed with two completely independent reviewers working solely from the written protocol. This limitation notwithstanding, field performance of the SBEI scoring method is good to excellent.
The basis of the SBEI scoring method on the hierarchy of controls supports its face validity. Furthermore, when used as pre- and post- intervention effectiveness measures as intended in this study, the baseline assessment of each area serves as its own reference or control, with the final evaluation metric being a measure of change . To the extent that a given area or process does not change fundamentally over the course of the intervention (e.g., gets replaced with an unrelated process or gets phased out), this strategy overcomes limitations inherent in comparing area ratings and scores cross-sectionally as well as longitudinally (as an intervention effectiveness measure).
We would hypothesize that cross-comparison of ratings and scores across production areas assessed and study sites would show corresponding relative levels of hazardous substance exposures. This has not been assessed in the current study because of the developmental stage of the instrument, technical and economic feasibility issues, and concerns about decreasing participation. With respect to feasibility, numerous agents would have to be sampled many times in each area assessed, which would involve considerable expense. In addition, requests to conduct such extensive sampling in the recruitment phase would be likely to further bias the sample of participating companies towards those with relatively good exposure control programs.
One approach to validation would be to obtain summary measures from multiple quantitative exposure measurements for each hazardous material in each area assessed. Measurements for each agent could then be transformed to a percent of a chosen set of Occupational Exposure Limits (OEL) (e.g., ACGIH, NIOSH, or OSHA). These summary percent OEL's could be averaged into an overall percent OEL across the range of agents present in each given area, paired with SBEI scores for each area, and analysed using standard correlational methods. We explored this possibility in the Wellworks-2 study, but found that routine monitoring was reported for only 14% of areas assessed . This showed that there was not enough company-collected quantitative exposure data available for validation studies even in large manufacturing worksites. We expected even less routine exposure monitoring to occur in small manufacturing settings, and thus did not explore this possibility in the Healthy Directions-Small Business study. Additionally, this demonstrates a gap in workplace exposure assessment practice that might be addressed in part through the application of more economical alternative strategies such as the approach described in this report.
Comparison of SBEI to Other Exposure Rating Schemes
Comparable assessment approaches to other hazardous exposures may also be feasible, such as ergonomic, safety, or other hazards. The development of a similar health and safety rating system has been reported for farm operations, wherein 'positive aspects' are balanced against 'negative aspects' for four different farm characteristics (operator attitude, operator characteristics, status of facility, and status of equipment) . A Site Rank Score is generated as the average ranking of the four characteristics. In this example, a very similar conceptual approach to the SBEI was generated independently for a different work context. Such rating schemes have broad applicability beyond manufacturing work settings.
SBEI also shares fundamental characteristics with control banding. The control banding approach has evolved from its origins in the pharmaceutical industry  to the sophisticated yet accessible web-based "COSHH Essentials" program designed by the U.K.'s Health and Safety Executive to assist small businesses in their efforts to understand and control risk http://www.coshh-essentials.org.uk. Control banding allows managers to use process knowledge, walkthroughs, toxicological data and other information sources to assign a job task to a "control band" based on risk potential. The band dictates the level of control needed for a particular operation. SBEI similarly promotes the integration of multiple hazard and exposure information sources, but also incorporates into its assessment the current control strategies for each operation. SBEI extends the control banding approach to create an integrated framework of risk and control assessment. The SBEI measures also extend control banding approaches in providing ratings and scores that are usable as intervention effectiveness evaluation measures.