Introduction

Assessment is one of the Fairness Commissioner's mandated roles under the Fair Access to Regulated Professions and Compulsory Trades Act, 2006 (FARPACTA) and the Regulated Health Professions Act, 1991 (RHPA) – collectively known as fair access legislation.

Assessment Cycle

One of the primary ways the OFC holds regulators accountable for continuous improvement is through the assessment of registration practices using a three-year assessment cycle.

Assessment cycles alternate between full assessments and targeted assessments:

• Full assessments address all specific and general duties described in the fair-access legislation.
• Targeted assessments focus on the areas where the OFC made recommendations in the previous full assessment.

Focus of this Assessment and Report

The 2016-2017 assessment of the College of Psychologists of Ontario (CPO or ‘the College”) is a full assessment.

The OFC’s detailed report captures the results of the full assessment. However, practices related to provision of information are excluded for regulators who have previously been assessed. For those regulators, these practices have been removed from the report.^[1] The assessment summary provides the following key information from the detailed report:

• duties that were assessed
• an overview of assessment outcomes for specific duty practices
• an overview of comments related to the general duty
• commendable practices
• recommendations

Top

Assessment Summary

Specific Duties

Specific duties assessed

The regulator has been assessed in all of the specific duties, with the exception of provision of information.

Comments

The regulatory body has demonstrated all of the practices in the following specific-duty areas:

• 2. Timely Decisions, Responses and Reasons
• 3. Internal Review or Appeal
• 8. Access to Documents

General Duty

Assessment method

The regulator selected the following method for the assessment of the general duty:

a.	OFC practice-based assessment (following the practices in the Assessment Guide)
b.	Regulator practice-based self-assessment (following the practices in the Assessment Guide)
c.	Regulator systems-based self-assessment (in which it explains systemically and holistically how it meets the general duty)

Principles assessed

The regulator has been assessed on all of the general duty principles: transparency, objectivity, impartiality and fairness.

Comments

The CPO implements measures to achieve transparent, objective, impartial and fair practices. The OFC and the CPO identified nine areas where the CPO needs to take further steps to demonstrate its obligations under the general duty (see the Recommendations section below.)

Commendable Practices

A commendable practice is a program, activity or strategy that goes beyond the minimum standards set by the OFC assessment guides, considering the regulatory body's resources and profession-specific context. Commendable practices may or may not have potential for transferability to another regulatory body.

The regulatory body is demonstrating commendable practices in the following areas:

Specific Duty

Training

1. Adding a standing agenda item entitled "Committee Development" to the Plenary Session of each Registration Committee meeting. As part of this item, the College’s staff share with Registration Committee members any new or emerging issues, such as outstanding OFC action plan items, information or resources provided by the OFC regarding fair registration requirements and/or practices.  This change ensures that the Registration Committee members are kept up-to-date on the College’s fair access agenda and about on-going actions taken by the College, thus enhancing the Registration Committee members’ capacity to make objective decisions.

General Duty

Transparency

1. Implementing a series of surveys to collect input from applicants, proctors, and examiners. The surveys are readily available to interested parties on the College’s website and enable contributors to provide comments on a variety of topics. This helps ensure a transparent and objective exam delivery process and to inform improvements in the College’s registration practices.
2. Implementing a comprehensive process for recording procedures that govern the registration process in policy documents. The process is recorded in College policy. It requires taking the following steps when developing a policy:
   1. conduct issue analysis and foundational research
   2. develop policy draft document using policy proposal template; and
   3. obtain approval by the registration and executive committees.
   This helps ensure that the College’s registration decisions are made on the basis of written criteria and procedures.
3. Making council meeting materials available to the public prior to the council meetings. The materials are posted in the “Resources” section of the College’s website.
4. Proactively informing applicants and interested stakeholders about upcoming changes. In December 2013, the College submitted to the government for approval changes to the registration and Quality Assurance (QA) regulations that would have a significant impact on these requirements. At that time, a notice was posted on the home page of the College’s website to alert potential applicants, members, and interested stakeholders that, once proclaimed, these changes would become effective immediately. In this way, even though these amendments were not proclaimed and did not become effective until 2015, applicants were provided significant advance notice of changes that could affect them.

Objectivity

1. Introducing a Language Fluency Policy. This policy allows applicants to meet the requirement in a number of ways. This includes:
   1. completing a psychology degree in a jurisdiction where English or French are the primary language of instruction and practice;
   2. achieving a minimum score on a standardized language fluency assessment
   3. providing relevant evidence to the Registration Committee, such as having worked as a psychologist or psychological associate in English or French.
   This policy provides guidance and clarity to registration staff and the Registration Committee to help determine if an applicant has met the requirement, thus enhancing the College’s capacity to make more expedient and objective registration decisions.

Fairness

1. Reducing registration and examination fees. The College amended its Bylaws to reduce fees for the following:
   1. annual registration fee for a member holding a certificate of registration authorizing supervised practice from $795 to $550; and
   2. two of the required examinations (Jurisprudence and Ethics Examination (JEE) and Oral Exam).
   The reduction in these fees will reduce the financial burden on eligible candidates and members holding a certificate authorizing supervised practice.
2. Implementing an Examination Accommodation Policy. The College had examination accommodation procedures in place, but developed a formal policy to help ensure that requests for accommodations are handled in a more expedient and fair manner. The Examination Accommodation Policy provides greater transparency and clarity to applicants regarding the process and implementation of exam accommodations. The new request forms required by this policy make requesting accommodations simpler for applicants. The Examination Accommodation Policy helps the College’s staff in:
   1. providing accommodation information to applicants more consistently and expediently;
   2. processing accommodation requests more efficiently; and
   3. enabling the administration of examinations with accommodations to proceed more smoothly.

Recommendations

The regulator can improve in the following area:

Specific Duty

Assessment of Qualifications

1. As part of reviewing the College’s assessment methods, verify that the criteria used in each assessment methods meet the following requirements:
   1. establish specific scores or grading scales used in the assessment methods that measure competencies or performance; and
   2. are expressed in measurable units.
   If the review identifies any need for improvement, implement appropriate actions and update guidelines for the College’s assessors ensuring that all the information on how to interpret criteria and how to assess evidence is included. (Practice 6.5)^[2]
2. Review all assessment methods to ensure they are objective, valid and reliable. The review should address the following:
   1. include psychometric scrutiny of all assessment methods;
   2. specify the purpose and objectives of each assessment method;
   3. identify what is being assessed, and the levels of competence required to pass the assessment; and
   4. if more than one assessment method is used to assess the same core competencies, determine relevance of each assessment method and/or assess whether there are any opportunities for streamlining the assessments. (Practice 6.7)^[3]
3. For all assessment methods, establish qualifications standards for assessors and select only those assessors that meet those standards:
   1. identify qualifications standards commensurate with the assessment tasks that are usually performed by the Registration Committee members and by proctors or examiners;
   2. verify whether the Registration Committee members and proctors and examiners meet those qualifications standards;
   3. if the review identifies any differences, identify actions for addressing them (i.e., training, bringing in additional expertise, etc.); and
   4. implement actions necessary for aligning the qualifications of the Registration Committee members and proctors and examiners with the assessment tasks performed by them. (Practice 6.9)

Training

1. In order to better inform decision-makers about characteristics or types of bias, and/or sources of bias, and/or circumstances that may compromise impartial registration decisions, continue to identify training opportunities for staff and Registration Committee members. This may include training offered through Ontario Regulators for Access Consortium and/or by the OFC. When the opportunities are identified, require staff and Registration Committee members to complete the training, as part of their continuing professional development. (Practice 7.3)

General Duty

Transparency

1. Undertake a review of the College’s standard communications letters (e.g. acceptance and refusal letters, requests for additional information) to ensure that they are written in plain language and are clear to applicants.
2. After completing the review, identify timeline(s) for the next periodic review(s) (i.e., how frequently the review will be undertaken and when).

Objectivity

1. Verify that decision makers have the policies and procedures that they need to do their job:
   1. review existing policies and procedures to ensure that they address all key decision points in the registration process (e.g., good character, providing accommodations, establishing terms and conditions, exempting from a requirement, approving requests for accommodations or any other requests, etc.);
   2. if the review finds any gaps in procedures or policy documents, identify the policies and procedures to be developed, and timelines for the completion of each; and
   3. as new policies and procedures are documented, provide decision makers with access to those tools and any other tools that will be created as a result of this work.
2. Develop a formal policy to consider and provide accommodations in cases where an applicant indicates that he or she cannot get the required documentation for reasons beyond his or her control.

Impartiality

1. In public documents, provide a clear rationale to explain what specific public safety or public interest issue(s) eliminating master’s degree registration intends to resolve. The rationale must be based in factual evidence illustrating that the change is necessary.
2. Identify and implement strategies to mitigate the threats to impartial evaluation of applicants’ competencies that are created by situations in which supervisors accept payments from the supervised applicants whose competencies they assess.

Fairness

1. Identify and implement acceptable alternatives to the academic requirement which will enable applicants to demonstrate their skills and knowledge (e.g., which would consider a combination of work experience obtained outside Ontario and education completed within and outside academic settings). Alternatively, clearly justify that the existing academic requirement is the only way to safely determine an applicant’s competencies to practice the profession.
2. In the public documents, clearly justify that establishing a doctoral degree as the standard for the College’s academic requirement is relevant and necessary to the practice of the profession. The justification should be based in factual evidence illustrating that the change is necessary.
3. Provide evidence of actions taken to address issues raised in the OFC’s “Internationally educated psychologists: a collaborative approach to examining the barriers to licensure in Ontario” report shared with the College earlier. Alternatively, provide supporting documentation of actions taken by the College to verify that its practices do not unjustifiably exclude or limit certain groups.

Assessment History

In the 2013-2014 assessment, the OFC identified 15 recommendations for CPO. Except for one, these recommendations were either fully implemented or the College took some actions to implement them. One recommendation has been carried forward to this report. Items that have yet to be completed were combined with recommendations identified during this assessment cycle.

1. Review the necessity and relevance of its Ontario-based supervised practice requirement. The review should address the following:
   1. identify the competencies underlying the requirement for supervised practice in Ontario;
   2. explore acceptable alternatives for applicants to demonstrate or develop these competencies; and
   3. implement alternatives or clearly justify why Ontario-based supervised practice is necessary and relevant to the practice of the profession. (Fairness)

This recommendation is not counted towards the total number of recommendations for this assessment cycle.

Top

Detailed Report^[4]

Specific Duty

2. Specific Duty — Timely Decisions, Responses and Reasons

RHPA, Schedule 2, s.20 (1)

1. If a regulator rejects an application, it gives written reasons to the applicant. [Fairness, Transparency]

Assessment Outcome

Demonstrated

3. Specific Duty — Internal Review or Appeal

RHPA, Schedule 2, s. 15, s. 17, s. 19, s. 22.3

1. The regulator provides applicants with an internal review of, or appeal from, registration decisions. [Fairness]

Assessment Outcome

Demonstrated

2. The regulator implements rules and procedures that prevent anyone who acted as a decision-maker in a registration decision from acting as a decision-maker in an internal review or appeal of that same registration decision. [Impartiality]

Assessment Outcome

Demonstrated

6. Specific Duty — Assessment of Qualifications

*Only applies to regulatory bodies that develop and administer their own exams.

RHPA, Schedule 2, s. 22.4(2)

2. The regulator communicates the results of qualifications assessment to each applicant in writing. [Transparency]

Assessment Outcome

Demonstrated

3. The regulator gives its assessors access to assessment criteria, policies and procedures. [Transparency]

Assessment Outcome

Demonstrated

4. The regulator shows that its tests and exams measure what they intend to measure*. [Objectivity]

Assessment Outcome

Demonstrated

OFC Comments

During the previous assessment cycle, the OFC recommended that the College review its assessment methods for objectivity, validity and reliability. In response to this recommendation, the College identified and implemented actions to address the recommendation. For example, the College commissioned an independent auditor to audit its Jurisprudence and Ethics Examination (JEE). As a result of this audit, the College revised its Conflict of Interest Policy as it relates to the JEE. It is important that the College continue its efforts in ensuring that its tests and exams measure what they intend to measure.

Suggestions for continuous improvement

Continue efforts taken to ensure that the test and exams measure what they intend to measure. On the website, publish information about preventative and/or corrective actions taken. Include relevant updates in the annual fair registration practices reports.

5. The regulator states its assessment criteria in ways that enable assessors to interpret them consistently. [Objectivity]

Assessment Outcome

Partially Demonstrated

OFC Comments

This practice requires that the regulator establish specific scores or grading scales for the assessment methods that measure competencies or performance. The OFC finds the College’s assessment criteria are not sufficiently specific. To address this, during the previous, 2013-2014 assessment cycle, the OFC recommended that the College examine all its assessment methods to ensure that they are objective, valid and reliable. As part of this review it would be important to verify validity and reliability of the assessment criteria for all assessment methods.

For example, the College uses interview as a method to conduct its Oral Exam. The College provides examiners with Rating Sheets to record their observations, rates, and the reasons for those rates. However, criteria for assigning rates seem to leave room for ambiguity and subjective interpretations, as they lack specificity. For example, there is no legend explaining how to assign rates “one” to “three.” It would be important to make the criteria more specific by providing examiners with instructions or a legend on how to assign rates for each dimension of the applicant’s professional performance.

Recommendations

1. As part of reviewing the College’s assessment methods:
   1. verify that the criteria used in each assessment method meet the following requirements:
      1. establish specific scores or grading scales used in the assessment methods that measure competencies or performance; and
      2. are expressed in measurable units.
   2. If the review identifies any need for improvement, implement appropriate actions and update guidelines for the College’s assessors ensuring that all the information on how to interpret criteria and how to assess evidence is included.
   This recommendation acknowledges the College’s actions completed to address recommendations made during 2013-2014 assessment cycle and combines the outstanding items with recommendations identified during this assessment cycle.

6. The regulator ensures that the information about educational programs that is used to develop or update assessment criteria is kept current and accurate. [Objectivity]

Assessment Outcome

Demonstrated

OFC Comments

When assessing academic credentials, the College relies on third parties to assess the level of education and to verify the status of the academic institutions that issued the credentials. As such, it may not be necessary for the College to extensively research the educational programs it assesses. To ensure consistency, the College refers to precedent decisions. However, it may be useful for the College’s assessors to broaden their knowledge of psychology education outside Canada and commonwealth jurisdictions.

Suggestions for continuous improvement

Identify and pursue opportunities for obtaining and accumulating knowledge about psychology education outside Canada and/or commonwealth jurisdictions. For example, this may include one or more of the following:

• undertaking research about similarities and differences of psychology education outside Canada to develop resources for assessors; and/or
• collaborating with other stakeholders in creating resources on psychology education outside Canada; and/or
• providing training on assessment methodology used by third-party assessment agencies and on how to interpret assessment reports prepared by those agencies; and/or
• attending assessment workshops offered by assessment agencies, such as World Education Services (WES) or American Association of Collegiate Registrars and Admission Officers (AACRAO).

7. The regulator links its assessment methods to the requirements/standards for entry to the profession or trade. [Objectivity]

Assessment Outcome

Partially Demonstrated

OFC Comments

This practice requires that the regulator link its assessment methods to the requirements for entry to the profession. The OFC finds that the College needs to take further actions to improve in this area. During the 2013-2014 assessment cycle, the OFC recommended that CPO identify all core competencies and develop objective criteria for assessing them. The OFC also recommended that once these steps are complete, the College should review its assessment methods to ensure that they are objective, valid and reliable. The CPO developed an action plan to respond to this recommendation and implemented actions to address the recommendation. However, further actions are still necessary. The recommendation identified below acknowledges the College’s actions completed already and identifies actions that are yet to be completed.

Recommendations

Review all assessment methods to ensure they are objective, valid and reliable. The review should include the following:

1. include psychometric scrutiny of all assessment methods;
2. specify the purpose and objectives of each assessment method;
3. identify what is being assessed, and the levels of competence required to pass the assessment; and
4. if more than one assessment method is used to assess the same core competencies, determine relevance of each assessment method and/or assess whether there are any opportunities for streamlining the assessments.

8. The regulator requires that assessors consistently apply qualifications assessment criteria, policies and procedures to all applicants. [Objectivity]

Assessment Outcome

Demonstrated

9. The regulator uses only qualified assessors to conduct the assessments. [Objectivity]

Assessment Outcome

Partially Demonstrated

OFC Comments

When assessing this practice, the OFC was seeking evidence of actions taken by the College to select its assessors in accordance with the qualification standards commensurate with the assessment tasks assigned to those assessors. Assessments of academic programs are performed by staff and Registration Committee members. Assessment staff is hired in accordance with the qualifications established in the positions’ job descriptions. The College maintains training materials to ensure that Registration Committee members are prepared to perform all forms of assessment conducted by the College. However, it is not obvious from the evidence received by the OFC how the College actually verifies the qualifications of the Registration Committee members. It is also unclear how the College verifies that examining team members are qualified to conduct those exams. The College should improve in this area (see Opportunities section below).

Recommendations

For all assessment methods, establish qualifications standards for assessors and select only those assessors that meet those standards:

1. identify qualifications standards commensurate with the assessment tasks that are usually performed by the Registration Committee members and by proctors or examiners;
2. verify whether the Registration Committee members, and proctors and examiners meet those qualifications standards;
3. if the review identifies any differences, identify actions for addressing them (i.e., training, bringing in additional expertise, etc.); and
4. implement actions necessary for aligning the qualifications of the Registration Committee members, and proctors and examiners with the assessment tasks performed by them.

10. The regulator monitors the consistency and accuracy of decisions, and takes corrective actions as necessary, to safeguard the objectivity of its assessment decisions. [Objectivity]

Assessment Outcome

Demonstrated

11. The regulator prohibits discrimination and informs assessors about the need to avoid bias in the assessment. [Impartiality]

Assessment Outcome

Demonstrated

OFC Comments

The College currently informs assessors about the need to avoid bias. To take these efforts further it may be useful to augment the existing policies with content addressing bias other than conflict of interest situations.

Suggestions for continuous improvement

Augment appropriate existing policies with content addressing cognitive biases (e.g., selective perception, stereotyping, preconceived notion, information bias, etc.)

12. The regulator implements procedures to safeguard the impartiality of its assessment methods and procedures. [Impartiality]

Assessment Outcome

Demonstrated

13. The regulator gives applicants an opportunity to appeal the results of a qualifications assessment or to have the results reviewed. [Fairness]

Assessment Outcome

Demonstrated

14. The regulator assesses qualifications, communicates results to applicants, and provides written reasons for unsuccessful applicants, without undue delay. [Fairness]

Assessment Outcome

Demonstrated

15. Regulators that rely on third-party assessments establish policies and procedures to hold third-party assessors accountable for ensuring that assessments are transparent, objective, impartial and fair. [Transparency, Objectivity, Impartiality, Fairness]

Assessment Outcome

Demonstrated

7. Specific Duty — Training

RHPA, Schedule 2, s. 22.4(3)

1. The regulator provides training for staff and volunteers who assess qualifications or make registration, internal review or appeal decisions. [Objectivity, Impartiality, Fairness]

Assessment Outcome

Demonstrated

Commendable practice

Adding a standing agenda item entitled "Committee Development" to the Plenary Session of each Registration Committee meeting. As part of this item, the College’s staff share with Registration Committee members any new or emerging issues, such as outstanding OFC action plan items, information or resources provided by the OFC regarding fair registration requirements and/or practices.  This change ensures that the Registration Committee members are kept up-to-date on the College’s fair access agenda and about on-going actions taken by the College, thus enhancing the Registration Committee members’ capacity to make objective decisions.

2. The regulator addresses topics of objectivity and impartiality in the training it provides to assessors and decision-makers. [Objectivity, Impartiality]

Assessment Outcome

Demonstrated

3. The regulator identifies when new and incumbent staff and volunteers require training and provides the training accordingly. [Objectivity, Impartiality, Fairness]

Assessment Outcome

Partially Demonstrated

OFC Comments

As part of its self-assessment of the general duty, the College identified that it can take further action in ensuring impartial decision-making by identifying and introducing additional training opportunities for staff and committee members. The OFC supports this commitment. While the College identified this Opportunity for Improvement under the General Duty assessment, the OFC believes that the actions to be taken to address this would also be necessary for meeting this specific duty practice. To avoid duplication, the OFC recorded this Opportunity for Improvement in this section.

Recommendations

In order to better inform decision-makers about characteristics or types of bias, and/or sources of bias, and/or circumstances that may compromise impartial registration decisions, continue to identify training opportunities for staff and Registration Committee members. This may include training offered through Ontario Regulators for Access Consortium and/or by the OFC. When the opportunities are identified, require staff and Registration Committee members to complete the training, as part of their continuing professional development.

8. Specific Duty — Access to Records

RHPA, Schedule 2, s. 16

1. The regulator provides each applicant with access to his or her application records. [Fairness]

Assessment Outcome

Demonstrated

2. If there is a fee for making records available, the regulator gives applicants an estimate of this fee. [Transparency]

Assessment Outcome

Demonstrated

3. If there is a fee for making records available, the regulator review the fee to ensure that it does not exceed the amount of reasonable cost recovery. [Fairness]

Assessment Outcome

Demonstrated

General Duty

RHPA, Schedule 2, S.22.2

Transparency

• Maintaining openness
• Providing access to, monitoring, and updating registration information
• Communicating clearly with applicants about their status

Assessment Outcome

The College implements measures to achieve transparency of its registration practices. This is evident from a variety of sources, including policy documents, examples of tools for decision-makers, and information posted on the College’s website. There are a few opportunities where the College should take further actions. One suggestion for continuous improvement and one recommendation are identified below.

Openness

The College uses effective strategies to create an open registration process. This includes the following:

• To help interested stakeholders, including the public and applicants, to understand how the registration process operates, the College publishes on its website:
  • information about the structure of accountability for registration functions; and
  • council meeting materials that are made available prior to the council meetings.
• To include public input in decisions about substantive registration changes, the College:
  • publishes information about public consultations in a distinct, visible section of its website; and
  • proactively seeks input on key assessment practices from applicants.

See specific examples of these actions in the Commendable practices section below.

Access

The College takes actions to ensure that applicants have all relevant information at the time and in the way needed to take actions appropriate to their individual circumstances. This includes organizing information on its website in a way that makes it easy to find. All the registration materials are posted in the distinctly marked section of the College’s website – “Resources.” For examples of commendable practices see Commendable practices section below.

Clarity

The College implements measures to communicate with applicants throughout the registration process. For example, the College has procedures in place for the following:

• informing applicants about how their application is progressing
• providing the reasons for all decisions taken during the registration process.

In the course of this assessment, the College and the OFC identified that the College should strengthen its process for reviewing its applicant letter templates (see Recommendations section below).

Commendable practice

1. Implementing a series of surveys to collect input from applicants, proctors, and examiners. The surveys are readily available to interested parties on the College’s website and enable contributors to provide comments on a variety of topics. This helps ensure a transparent and objective exam delivery process and to inform improvements in the College’s registration practices.
2. Implementing a comprehensive process for recording procedures that govern the registration process in policy documents. The process is recorded in the College’s policy. It requires taking the following steps when developing a policy:
   1. conduct issue analysis and foundational research;
   2. develop policy draft document using policy proposal template; and
   3. obtain approval by the registration and executive committees.
   This helps ensure that the College’s registration decisions are made on the basis of written criteria and procedures.
3. Making council meeting materials available to the public prior to the council meetings. The materials are posted in the “Resources” section of the College’s website.
4. Proactively informing applicants and interested stakeholders about upcoming changes. In December 2013, the College submitted to the government for approval changes to the registration and Quality Assurance (QA) regulations that would have a significant impact on these requirements. At that time, a notice was posted on the home page of the College’s website to alert potential applicants, members, and interested stakeholders that, once proclaimed, these changes would become effective immediately. In this way, even though these amendments were not proclaimed and did not become effective until 2015, applicants were provided significant advance notice of changes that could affect them.

Suggestions for continuous improvement

Consider publishing and archiving submissions or a summary of submissions received in response to the College’s consultations with the public.

Recommendations

1. Undertake a review of the College’s standard communications letters (e.g. acceptance and refusal letters or requests for additional information) to ensure that they are written in plain language and are clear to applicants.
2. After completing the review, identify timeline(s) for the next periodic review(s) (i.e., how frequently the review will be undertaken and when).

Objectivity

• Designing criteria and procedures that are reliable and valid
• Monitoring and following up threats to validity and reliability

Assessment Outcome

The College takes some measures to achieve objective registration practices. This is evident from a variety of sources, including policy documents, examples of tools for decision-makers, and information posted on the College’s website. The OFC and the College identified further actions for improvement.

When assessing how the College meets its general duty obligations related to objectivity, the OFC was looking for documented evidence of written guidelines for staff and volunteers who make decisions including the following:

1. issuing letters of eligibility for applicants who didn’t complete supervised practice experience to permit them to take registration exams
2. re-issuing the certificates for supervised practice in instances when applicants were unsuccessful in the registration exams
3. providing applicants who cannot provide the required documentation with alternative documentation options.

These policies and guidelines should enable the decision-makers to make consistent and accurate decisions. The decisions must be based on evidence that directly demonstrates how the applicant meets or doesn’t meet the requirements and be confined only to evidence that is specifically related to the decision itself. The OFC was also seeking to identify measures taken by the College to monitor the consistency and accuracy of its decisions. The OFC identified the following measures (see Reliability and Validity sections below.)

Reliability

To promote the reliability of its registration decisions, the College:

• expresses the registration criteria in measurable units and/or establishes specific conditions applicants must meet to satisfy a registration requirement; and
• has a process to review and update statements describing registration requirements and criteria for clarity.

Validity

To monitor accuracy and consistency of registration decisions, the College:

• implements a policy that requires regular internal audits of the College’s statutory committees. During such an audit of the Registration Committee a random sample of past registration decisions may be reviewed; and
• prepares an internal audit report as a result of the audit. The report identifies issues and makes recommendations for how these issues may be addressed.

The OFC also found that the College could take further actions to meet its obligations related to objectivity. This includes one suggestion for continuous improvement and two recommendations (see sections below).

In particular, it is unclear to the OFC how the College provides its decision-makers with the information and tools they need to make decisions about issues identified at the top of this section (see items a) to c)). The OFC notes that within the context of the College’s registration process, it may not be practical for the College to have all of its procedures recorded in manuals and policy documents. It seems that in some instances it may be more practical for the College to deal with issues using an ad-hoc approach rather than creating formal policies for addressing one-off situations. If that is so, the potential for subjective decision-making may be mitigated by introducing one overarching policy that would identify when it would be appropriate to address an issue using an ad-hoc approach, and what steps to take and/or considerations to make when applying such an approach. Otherwise, it would be important to develop more specific criteria and procedures to address a) to c) and any other registration decisions made that are not yet addressed by written criteria and procedures. As part of its self-assessment, the College identified the need to develop a policy to address c). The OFC supports this commitment.

With regards to maintaining consistency in the registration decisions, the OFC was looking for evidence of actions taken to ensure that decision-makers consistently apply criteria, policies and guidelines when making decisions that are non-assessment decisions, such as about “past-conduct” of applicants or about issues identified at the top of this page, including a) to c).

The College uses Individual Review Forms to record assessment findings, decisions, and the reasons for the decisions regarding applicants’ academic programs. However, it is unclear how the College ensures that it uses appropriate evidence for determining whether applicants meet other registration requirements and how it achieves consistency in non-assessment decisions. The OFC finds that the College needs to improve in these areas (see Recommendations section below).

Commendable practice

Introducing a Language Fluency Policy. This policy allows applicants to meet the requirement in a number of ways. This includes:

1. completing a psychology degree in a jurisdiction where English or French are the primary language of instruction and practice;
2. achieving of a minimum score on a standardized language fluency assessment; and
3. providing relevant evidence to the Registration Committee, such as having worked as a psychologist or psychological associate in English or French

This policy provides guidance and clarity to registration staff and the Registration Committee to help determine if an applicant has met the requirement, thus enhancing the College’s capacity to make more expedient and objective registration decisions.

Suggestions for Continuous Improvement

As part of an internal audit, include procedures to identify any potential factors that may compromise the consistency of the College’s registration decisions. Develop tools or implement procedures to mitigate those risks.

Recommendations

1. Verify that decision makers have the policies and procedures that they need to do their job:
   1. review existing policies and procedures to ensure that they address all key decision points in the registration process (e.g., good character, providing accommodations, establishing terms and conditions, exempting from a requirement, approving requests for accommodations or any other requests, etc.)
   2. if the review finds any gaps in procedures or policy documents, identify the policies and procedures to be developed, and timelines for the completion of each.
   3. as new policies and procedures are documented, provide decision makers with access to those tools and any other tools that will be created as a result of this work.
2. Develop a formal policy to consider and provide accommodations in cases where an applicant indicates that he or she cannot get the required documentation for reasons beyond his or her control.

Impartiality

• Identifying bias, monitoring, and taking corrective action
• Implementing strategies

Assessment Outcome

The College implements measures to achieve impartial registration decisions. This is evident from a variety of sources, including policy documents, examples of tools for decision-makers, and information posted on the College’s website. The OFC and the College identified further actions for improvement. This includes one Suggestion for Continuous Improvement and two recommendations (see respective sections below).

Identification of Bias

The OFC identified that the College takes the following steps:

• documenting and informing its decision-makers about the need to avoid bias in the registration decisions; and
• documenting and informing its decision-makers about what constitutes a conflict of interest and what decision-makers should do when they find themselves in the position of a conflict of interest.

While these are important actions to take, the OFC has a continued concern about the College’s attempts to discontinue master’s level registration. Comments below review this issue from an impartiality perspective. Comments in the fairness section of this report address this issue from a fairness perspective.

From an impartiality perspective, when making substantive changes to the registration policy or requirements, it is important that a decision about such a change meets the following conditions:

1. it is not improperly influenced by internal or external factors;
2. it is made in the public interest and it resolves a specific public interest or a public safety issue; and
3. it is made following considerations of positive and negative impacts the decision would have on those affected by it.

In the course of this assessment, the OFC was seeking to verify that the decision to discontinue master’s level registration meets these conditions. While the College provided documentation showing that the analysis was undertaken, the analysis itself is lacking factual evidence and does not address all three conditions. In particular, it is unclear to the OFC how the College arrived at the decision that discontinuing master’s level registration is in the public interest. It is also unclear what specific public interest or safety issue this change intends to resolve.

Setting a doctoral degree as the standard for entry to practice on the basis of analysis that does not provide factual evidence that the change would be necessary would be contrary to the principles of impartiality and fairness. Further actions are necessary to improve in this area (see Recommendations section below). These actions may have to be combined with actions identified for improving Fairness as they address similar areas (See General Duty – Fairness section).

Implementing strategies

The OFC identified that the College takes the following steps:

• conducting internal audits of statutory committees’ work. If an audit identifies any issues, taking steps to address those issues;
• requiring that decisions regarding an application be unanimous. This is done to reduce the potential for any panel member to have a disproportionate impact on the decision. All decisions must be grounded in the defined criteria; and
• implementing procedures requiring that when necessary, the Director of Registration or Chair of the panel can remind other panel members of their obligations to adhere to written criteria and procedures.

While these are important actions to take, the OFC has an ongoing concern about them. In fact, it is not uncommon for applicants completing a supervised practice requirement to have to pay for supervision. This arrangement not only poses a serious burden on applicants as they struggle to cover living expenses at the same time, but also creates situations in which supervisors accept payments from the trainees whose competencies they assess. This could potentially influence, or appear to influence, the supervisor’s ability to provide an objective and impartial evaluation of the trainee’s competence. This was raised with the College previously. In its registration materials, the College provides guidelines identifying the conditions in which payments would be appropriate. However, it is unclear how this addresses the issue. Further actions are necessary to safeguard impartiality of the College’s registration decisions (see Opportunities for improvement section below).

Suggestions for Continuous Improvement

Augment existing policies addressing bias with content addressing cognitive biases (e.g., selective perception, stereotyping, preconceived notion, information bias, etc.)

Recommendations

1. In public documents, provide a clear rationale to explain what specific public safety or public interest issue(s) eliminating master’s degree registration intends to resolve. The rationale must be based in factual evidence illustrating that the change is necessary.
2. Identify and implement strategies to mitigate the threats to impartial evaluation of applicants’ competencies that are created by situations in which supervisors accept payments from the supervised applicants whose competencies they assess.

Fairness

• Ensuring substantive fairness
• Ensuring procedural fairness
• Ensuring relational fairness

Assessment Outcome

The College implements measures to promote fair registration practices. This is evident from a variety of sources, including policy documents, examples of tools for decision-makers, and information posted on the College’s website. Examples of effective measures are provided below in this section. The OFC identified that the College should take further actions to improve (see Recommendations section below). These actions may have to be combined with actions identified for improving Impartiality and Assessment of Qualifications procedures as they may require similar responses. (See General Duty – Impartiality and Specific Duties – Assessment of Qualifications sections).

Substantive fairness

Substantive fairness requires that the regulators’ registration requirements are relevant and necessary to the practice of the profession. The onus is on the regulatory body to demonstrate this. To do this, the College should show that it does one of the following

1. implements acceptable alternatives for meeting the requirement; or
2. shows that there is no other way to safely determine an applicant’s competencies to practise the profession or trade.

Currently, when an international applicant’s education is not found to be equivalent to Ontario’s psychology education, a combination of an applicant’s working experience and education is not taken into consideration to determine equivalence. This approach may potentially exclude some qualified applicants. The College didn’t provide sufficient evidence that there is no other way to safely assess applicants’ competencies other than assessing program equivalency alone.

In early 2016, the OFC commented on the College’s Implementation Plan Regarding Change to Entry-to-Practice Requirements (“Plan”). The Plan included provisions for developing a mechanism for evaluating internationally trained applicants’ competencies for substantive similarities to a program accredited by the Canadian Psychological Association. The OFC supported the College’s movement towards developing such a mechanism. It is important that the mechanism enable the College to consider a combination of work experience and of formal and non-formal education when assessing applicants’ competencies, rather than focusing on the comparability of an academic credential alone (see Recommendations section below).

In 2016, the College consulted stakeholders on its proposal to close the Psychological Associate class while maintaining a doctoral degree as the standard for entry to practice for Canadian and American educated applicants. The OFC responded to the consultation expressing its concerns that setting the doctoral degree as the standard for entry to practice will restrict access to the psychology profession in Ontario for Canadian and American educated applicants. During this assessment, the OFC was seeking evidence to address this concern. However, the concern remains. The OFC didn’t find evidence clearly justifying that establishing a doctoral degree as the standard for the College’s academic requirement is relevant and necessary to the practice of the profession. The College does not adequately explain the rationale for this proposed change. While the College provided documentation showing that the analysis was undertaken, the analysis itself is lacking factual evidence. It is not obvious from the analysis and evidence obtained that the only way for the College to safely assess applicants’ competencies is to admit only applicants that have a doctoral degree in Psychology. Proceeding with the change without concrete and supporting evidence would be contrary to the principle of fairness (see Recommendations section below).

Furthermore, during the 2013-2014 assessment cycle, the OFC identified that the College does not clearly justify why the supervised practice requirement must be completed in Ontario and made a recommendation to the CPO (see Recommendations section below). The CPO developed an action plan to respond to this recommendation. However, not all actions identified in the action plan are yet completed. The OFC understands that this delay is due to other competing priorities and limited resources at the CPO.

Procedural Fairness

To promote procedural fairness, the College:

• Periodically reviews its compliance with its registration policies and procedures.
• Conducts internal audits.
• Monitors that the Registration Committee members follow the Policy and Procedures Manual for the Registration Committee.

While these are important actions to take, procedural fairness also requires that the regulator can show that its registration procedures do not unjustifiably exclude or limit certain groups, such as internationally trained applicants. Following the previous assessment, the OFC conducted research into how internationally trained psychologists perceive the College’s registration process and found that many perceive the process as very challenging. The OFC recorded its findings in an internal report and shared a copy of this report with the College. During this assessment, the OFC requested that the College provide evidence of actions taken by the College to explore or address issues raised in that report. The OFC has not received such evidence yet (see Recommendations section below).

Relational Fairness

The College makes commendable efforts in taking applicants’ circumstances into consideration by accommodating special needs in examinations, including:

• making the testing centres accessible;
• allowing additional time; and
• allowing the use of an approved reader, scribe or sign language interpreter
• providing options for presenting the exam in braille, large print and audio formats.

The College has a process to consider individual circumstances to allow applicants to proceed to next steps in the registration process.

Commendable practice

1. Reducing registration and examination fees. The College amended its Bylaws to reduce fees for the following:
   1. annual registration fee for a member holding a certificate of registration authorizing supervised practice from $795 to $550; and
   2. two of the required examinations (JEE and Oral Exam).
   The reduction in these fees will reduce the financial burden on eligible candidates and members holding a certificate authorizing supervised practice.
2. Implementing an Examination Accommodation Policy. The College had examination accommodation procedures in place, but developed a formal policy to help ensure that requests for accommodations are handled in a more expedient and fair manner. The Examination Accommodation Policy provides greater transparency and clarity to applicants regarding the process and implementation of exam accommodations. The new request forms, required by this policy, make requesting accommodations simpler for applicants. The Examination Accommodation Policy helps the College’s staff in:
   1. providing accommodation information to applicants more consistently and expediently;
   2. processing accommodation requests more efficiently; and
   3. enabling the administration of examinations with accommodations to proceed more smoothly.

Recommendations

1. Identify and implement acceptable alternatives to the academic requirement which will enable applicants to demonstrate their skills and knowledge (e.g., which would consider a combination of work experience obtained outside Ontario and education completed within and outside academic settings). Alternatively, clearly justify that the existing academic requirement is the only way to safely determine an applicant’s competencies to practice the profession.
2. In the public documents, clearly justify that establishing a doctoral degree as the standard for the College’s academic requirement is relevant and necessary to the practice of the profession. The justification should be based in factual evidence illustrating that the change is necessary.
3. Provide evidence of actions taken to address issues raised in the OFC’s report shared with the College earlier. Alternatively, provide supporting documentation of actions taken by the College to verify that its practices do not unjustifiably exclude or limit certain groups.

Recommendations

The recommendations made during 2013-2014 are worded as follows:

1. Review the necessity and relevance of its Ontario-based supervised practice requirement.
   1. Identify the competencies underlying the requirement for supervised practice in Ontario;
   2. Explore acceptable alternatives for applicants to demonstrate or develop these competencies; and
   3. Implement alternatives or clearly justify why Ontario-based supervised practice is necessary and relevant to the practice of the profession.

Top

Background

Assessment Methods

Assessments are based on the Registration Practices Assessment Guide: For Regulated Professions and Health Regulatory Colleges. The guide presents registration practices relating to the specific duties and general duty in the fair access legislation.

A regulatory body’s practices can be measured against the fair access legislation’s specific duties in a straightforward way. However, the general duty is broad, and the principles it mentions (transparency, objectivity, impartiality and fairness) are not defined in the legislation.

As a result, the specific-duty and general-duty obligations are assessed differently (see the Strategy for Continuous Improvement).

Specific Duties

The OFC can clearly determine whether a regulatory body demonstrates the specific-duty practices in the assessment guide. Therefore, for each specific-duty practice, the OFC provides one of the following assessment outcomes:

• Demonstrated – all required elements of the practice are present or addressed
• Partially Demonstrated – some but not all required elements are present or addressed
• Not Demonstrated – none of the required elements are present or addressed
• Not Applicable – this practice does not apply to the CPO’s registration practices

General Duty

Because there are many ways that a regulatory body can demonstrate that its practices, overall, are meeting the principles of the general duty, the OFC makes assessment comments for the general duty, rather than identifying assessment outcomes. For the same reason, assessment comments are made by principle, rather than by practice.

For information about the OFC's interpretations of the general-duty principles and the practices that the OFC uses as a guideline for assessment, see the OFC's website.

Commendable Practices and Recommendations

Where applicable, the OFC identifies commendable practices or recommendations for improvement related to the specific duties and general duty.

Sources

Assessment outcomes, comments, and commendable practices and recommendations are based on information provided by the regulatory body. The OFC relies on the accuracy of this information to produce the assessment report. The OFC compiles registration information from sources such as the following:

• Fair Registration Practices Reports, audits, Entry-to-Practice Review Reports, annual meetings
• the regulatory body's:
  • website
  • policies, procedures, guidelines and related documentation templates for communication with applicants
  • regulations and bylaws
  • internal auditing and reporting mechanisms
  • third-party agreements and related monitoring or reporting documentation
  • qualifications assessments and related documentation
• targeted questions/requests for evidence that the regulatory body demonstrates a practice or principle

For more information about the assessment cycle, assessment process, and legislative obligations, see the Strategy for Continuous Improvement.

Top

References

1. ^ These include: all practices from Information for Applicants, practice 3 from Internal Review and Appeals, practice 1 from Information on Appeal Rights, practice 1 from Documentation of Qualifications, practice 1 from Assessment of Qualifications, practice 2 from Access to Records, and practices 4-11 from Transparency of the Registration Practices Assessment Guide.
2. ^ This recommendation acknowledges the College’s actions completed to address recommendations made during 2013-2014 assessment cycle and combines the outstanding items with recommendations identified during this assessment cycle.
3. ^ This recommendation acknowledges the College’s actions completed to address recommendations made during 2013-2014 assessment cycle and combines the outstanding items with recommendations identified during this assessment cycle.
4. ^ Please note: Suggestions for continuous improvement appear only in the detailed report. Suggestions for improvement are not intended to be recommendations for action to demonstrate a practice, but are made solely to provide suggestions for areas that a regulatory body may consider improving in the future.