4.  THE REVIEW PROCESS

Process Overview

Entry-to-Practice Reviews consist of four key steps. The time to complete a review will vary, depending on the scope of the review and the chosen methodology.

Step 1: Create and approve the project plan

This first step is to create and approve a project plan. In a mandatory review, the project plan should contain each of the elements listed below along with any others the regulator wishes to add.

	The Project Plan
Objectives	What is the review intended to accomplish? How will the findings be used?
Scope	What topics will the review cover in depth and what will be covered in a summary way?
Review questions	What are the specific questions the review will be designed to answer?
Data collection plan	What information will be collected, and why? From what sources? Whose input will be sought and why? How will it be obtained? Will an outside facilitator or researcher be retained?
Project governance	Who is accountable for the review? Will there be an oversight or advisory committee?
Project management	What are the key tasks, products, milestones, timelines and resources? Who are the team members, and what are their individual roles?

Step 2: Collect and analyze data

What information should be collected?

Much of an Entry-to-Practice Review will be spent collecting and analyzing data. Work already done for an earlier Fair Registration Practices Report or Audit Report should make some of the data collection much easier. The following types of data will be helpful.

TYPE OF DATA	PURPOSE
Governing statute, regulations, bylaws, standards, guidelines, policies and procedures	To identify registration requirements that must be analyzed for necessity and relevance To identify registration practices that must be analyzed for transparency, objectivity, impartiality and fairness
Qualitative data (through interviews, focus groups, surveys, or questionnaires)	To learn about positive and negative experiences and ideas for improvement
Statistics, appeals, applicant complaints	To identify trends or problems in general or for certain classes of applicants; flag issues to probe
Contextual information about the environment in which you operate (“environmental scan”)	To understand trends and emerging issues

It could also be helpful to get the following types of data, depending on the scope of the review.

TYPE OF DATA	PURPOSE
Information on other professions or jurisdictions	To compare with and learn from comparable organizations (supplementing any research that may be provided by the Fairness Commissioner)

Whose input should be gathered?

One of the guiding principles for Entry-to-Practice Reviews is that they be participatory. In a mandatory review, input should be gathered from:

• Registrar, Registration Committee and staff
• Members and current applicants
• Applicants who were unsuccessful or withdrew
• Qualifications assessment agencies (including affiliated national bodies).

Be sure to include a variety of members and applicants, including internationally trained ones, in order to get a range of perspectives on registration requirements and practices.

Depending on the nature and scope of the review, regulators are also encouraged to ask for input from one or more of the following:

• Professional associations
• Similar regulatory bodies in other jurisdictions
• Other Ontario regulatory bodies in related professions
• Organizations that support internationally trained applicants
• Educational institutions that offer programs needed for the profession
• Programs that help internationally trained applicants to upgrade their qualifications
• Employers that offer workplace or clinical experience required for registration
• Ministries and other government bodies.

How should the data be analyzed?

The purpose of analyzing the data is to determine which requirements and practices work well and which ones will be the subject of recommendations for change. Analysis is often the most difficult part of the review process. The extent of analysis depends on which topics the Commissioner specifies for in-depth or summary analysis and which topics the regulator adds.

For topics identified for in-depth review, the final report should include a thoughtful response to each review question based on what the data, consultations and analysis show. For topics identified for summary analysis, a brief paragraph in the report will suffice.

When should you seek outside help?

In most cases, it should be possible to complete the review without third-party help. When the collection and analysis of data will be extensive, and the regulatory body has the resources, it may wish to hire an external consultant to help in specific parts of the review. Under the regulatory body's direction, a consultant could prepare the project plan for approval, design surveys or consultation questions, conduct focus groups or facilitate meetings, and draft or edit the final report. In addition, regulators who have conducted reviews can be a source of ideas and tips on how to conduct a review, with or without the help of an external consultant.

Step 3: Make recommendations and an implementation plan

Determine recommendations

A key part of an Entry-to-Practice Review is recommending which registration requirements or practices need to be modified and the nature of the changes. In some cases, the modified requirement or practice may be more or less stringent than the one that currently exists. In other cases it will simply reflect a different approach. In all cases, the change should be clearly justified based on an assessment of the considerations set out for Entry-to-Practice Reviews in the legislation. Regulators conducting reviews must ensure that any new or revised requirement or practice does not create unnecessary barriers for applicants.

Develop an implementation plan

It is also essential to develop an implementation plan to include in the final report. The implementation plan should set out measurable goals for each recommended change, timelines for achieving the goals, the resource allocation, and responsibility within the organization for meeting the goals and evaluating progress.

Implementation of some changes may require action by the government or legislature. In that case, the regulatory body should take all steps possible to advocate for the change, with help from the Fairness Commissioner.

Step 4: Write, approve and file the report

The final step in an Entry-to-Practice Review is to write, approve and file the report.

Write the report

Reports for mandatory reviews should contain the components listed below and cover the topics listed in the checklist in Appendix A.²²

Approve the report

FARPA requires that the report include a statement by a person in authority certifying that all the information required has been provided and is accurate. ²³ Ideally, the report should be approved by the regulator's Registration Committee and board/council to ensure there is support for the recommendations and a commitment to implement the changes.

File the report

Reports from mandatory reviews must be filed with the Fairness Commissioner. For regulators governed by the RHPA, ²⁴ the filing deadline is 30 days from completion of the review. Regulatory bodies governed by FARPA must file by the date specified by the Commissioner. ²⁵ In exceptional circumstances, the Commissioner may extend a date for a regulatory body governed by FARPA. The Commissioner encourages regulators conducting voluntary reviews to also file their reports with the OFC.

Normally, the Commissioner will specify that a report from a regulator governed by FARPA must be filed within 30 days after the time specified for completing the review. This will provide consistency in the timing for health and non-health professions.

Make the report public

Non-health regulators must make the filed report available to the public. ²⁶ Although there is no similar requirement for health professions, it is a good practice to make the reports publicly available in the spirit of transparency. The simplest way to make the report available to the public is by posting it on the regulator's website.

---