Construction and validation of an instrument for the assessment of care provided to people with suicidal behavior

ABSTRACT OBJECTIVE To develop and validate an instrument for evaluating primary health care professionals’ assistance to people with suicidal behavior. METHODS This was a methodological study, which began with a literature review, followed by the elaboration of an instrument. In its first version, the instrument had 34 items, divided into four domains: “professional characterization,” “professional perception ” “professional knowledge/abilities,” and “organization of the care network.” Contents were validated using the Delphi method. Semantic analysis was performed by college-educated primary health care professionals in greater and lesser strata of ability. For internal consistency analysis, Cronbach’s alpha coefficient was calculated. The study was conducted between January and December 2017. RESULTS After four Delphi rounds, the instrument was successfully validated. In its final form, it is comprised of 50 items, divided into five domains: “professional characterization,” “professional sensibility,” “professional experience,” “professional knowledge/abilities,” and “organization of the care network.” Questions belonging to the last four domains have answers on a five-point Likert scale. In the semantic analysis, 93.6% of the evaluations were “good” and “very good.” The instrument’s general Cronbach alpha was 0.90. CONCLUSIONS The final version of the instrument was able to fulfill its objectives. It is useful as a support for epidemiological research and planning of health actions. The evaluation of professional approaches to suicidal behavior is crucial for the organization of suicide assistance services in primary health care, and for the integration of services provided by different care units.


INTRODUCTION
Suicidal behavior is typified into four different categories: suicidal ideation, suicide planning, suicide attempt, and suicide. Suicide stems from an association between biological, genetic, psychological, social, environmental, and situational factors. No single factor can provide a complete causal explanation for suicide 1 .
This behavior is often related to an individual's emotional impossibility of recognizing alternatives for the resolution of conflicts and sufferings. There are several factors associated with the risk of suicide, including disabling physical illnesses, mental illnesses, abuse of alcohol and other drugs, as well as socioeconomic and family issues 1,2 .
Although it is considered preventable, worldwide more than 800,000 people commit suicide every year. It is estimated that for each effective act of suicide, there are more than 20 attempts 1 . Brazil is the eighth country in absolute number of suicides, with almost 12,000 annual deaths (approximately 32 per day) 1 . From 2004 to 2014, there was a higher concentration of suicides among individuals aged 20 to 49 years, as well as a 47% increase in the number of suicides among older people in the 60-69 age group 3 .
In a survey conducted in England, 91% of the sample had consulted a doctor at their primary health care provider (PHC) at least once in the year prior to the suicide act; almost 50% had their last consultation with this professional in the month prior to suicide, and one sixth in the week prior 4 .
The current suicide and attempted suicide rates point to a serious public health problem, which requires preventive action. In view of this scenario, Brazil has instituted National Guidelines for the Prevention of Suicide. The document recognizes the importance of preventative interventions, highlighting measures such as the organization of care networks for people at risk of suicide, as well as the development of data collection and analysis methods that enable the sharing of information and knowledge. It also suggests the promotion of continuing education for health professionals, including in the PHC 5 . The training of PHC teams to identify, approach, manage and refer people with suicidal behavior is an important means of prevention, since these professionals' proximity and bonds with the population are the main resources of the primary health care system 6 .
An understanding of these professionals' practice is fundamental, and having instruments that provide valid and reliable measures to identify their competencies is a relevant factor for building guidelines and organizing the care network 7,8 . Identifying the knowledge and practice of professionals working in the PHC system allows for the design and development of more effective training, subsidizing the planning of health interventions.
In the literature review, we found no studies on instruments to evaluate the care for suicidal people provided by these professionals. In view of the above, the objective of the study was to construct and validate an instrument for assessing the care provided by college-educated PHC professionals to people with suicidal behavior.

METHODS
This was a methodological study developed in four phases, describing the instrument's construction process, the validation of its contents by specialists, the semantic analysis of its questions, and the evaluation of its internal consistency. The research was conducted between January and December 2017.
In the construction phase, the scientific literature was reviewed. Descriptors used in the search were: health personnel, suicide, suicide attempted, scales, primary health care, surveys and questionnaires, and delivery of health care, in English and Portuguese languages. The Web of Science, PubMed, and Medline databases were consulted. Inclusion criteria considered complete publications in national and international journals, starting from 1990. After reading of the abstracts, studies that did not fit this criteria were excluded. All selected articles were read in two stages: I. search for relevant suicide prevention aspects, and confirmation of inclusion in the study; and II. selection of information on the subject for inclusion in the instrument. A total of 159 articles was found; 151 were excluded. Thus, the elaboration of the instrument was based on the consultation of eight articles, as well as on the recommendations and protocols of the World Health Organization 1,6 , the Ministry of Health a,b , and the Brazilian Association of Psychiatry c . The selected articles referred to instruments for the assessment of professional attitudes toward suicidal behavior, but none focused on the PHC system or on the assessment of professional care itself.
Based on the literature review, the first version of the instrument was elaborated, consisting of 34 items, among which eight concerned professional characterization and 26 were aimed at the investigation of professional sensibility and knowledge, as well as the organization of the care network. Items were divided into four domains: "professional characterization," "professional sensibility," "professional knowledge/abilities," and "organization of the care network." Items regarding the knowledge and practice of professionals were answered according to a five-point Likert scale, with five representing maximum agreement, and 1 representing minimum agreement.
In the second phase, the content of the instrument was validated by specialist judges. Participation in the research as a judge entailed the following requirements: I. to work in the area of mental health or in the PHC network, having been appointed by the Ministry of Health, State Health Department of Mato Grosso do Sul, Municipal Health Secretariat of Campo Grande, Brazilian Association of Psychiatry, or Oswaldo Cruz Foundation (Fiocruz); or, II. to have scientific publications in the area of mental health or PHC. Specialists who had been working in their respective functions for less than a year, or who did not respond to the invitation to participate within the established deadline, were excluded. The Delphi method was used, with specialists being consulted about the instrument' s proposals. Consensus required at least two Delphi rounds 9 .
The survey began with seven judges. Six answered it 10 , remaining for four Delphi rounds. The instrument was sent via e-mail or traditional mail for item-by-item evaluation. Each judge gave their opinion on whether items should remain as they were, inserted, changed, or removed. Judges also provided their general evaluation of the instrument. Answers were regarded as stable when there was an agreement of at least 70% between judges 11 . The researchers proposed a duration of up to 21 days for each round, including 14 days for analysis and feedback by the group of judges, and a maximum of seven days for a response by the researcher, with a new version of the instrument attached, at which point the new round could begin. Answers and analysis were provided by the judges in an Excel ® spreadsheet. Whether or not consensus was reached, the outcome of the previous round was reported to the group during the evaluation of the new iteration (which included the proposed changes).
This process led to a final version containing 50 items, with 14 belonging to the "professional characterization" domain, and 36 belonging to the four other domains: "professional sensibility," "professional experience," "professional knowledge/abilities," and "organization of the care network." It is worth noting that the domain "professional characterization" differs from others, since it could not be answered according to a Likert scale. Analyzing the internal consistency of its items was not possible. However, according to the body of judges, this domain is extremely important for the instrument's objectives. This is because it is not limited to the characterization of sociodemographic profile, but evidences data referring to professionals' training and practices in dealing with risk factors for suicidal behavior. The researchers agreed with the judges, so the domain was kept in the instrument's final version.
Content validation was followed by the research's third phase: semantic analysis. This step examined whether the target population -including strata of greater or lesser ability 10  The inclusion criteria were: I. to work in the municipal health network performing a function within the professional's area of graduation; and II. to have an electronic address for receiving the link to access the evaluation. Professionals who had been in PHC for less than six months or who did not fill the questionnaires until the end of the process were excluded from the survey.
The open source software LimeSurvey was used. Participants accessed a web page where they were able to fill out the informed consent form (TCLE), the judge-validated instrument, and the semantic analysis questionnaire. The page was sent individually, via e-mail, to 283 professionals, who had ten days to participate. Fifty-three professionals participated, among whom three were excluded because they had been working in the PHC system for less than six months. Fifty (50) remained.
Cronbach's alpha was calculated from answers given by the professionals, according to a Likert scale, to items in four domains. The items were related to assistance provided to people with suicidal behavior, the study's object. This test checks responses' pattern consistency as well as the instrument's reliability, and its result should be equal to or greater than 0.70 12 . Descriptive statistics were performed using SPSS software version 24.0. Results are presented descriptively and in tables.

RESULTS
Six judges participated in the application of the Delphi technique. Their characterization is shown in Table 1. In the first round, a consensus of 70% was obtained for 21 (61.7%) of the Box. Changes made in the instrument (assessment of care provided by college-educated professionals of primary health care to people with suicidal behavior). Campo Grande, state of Mato Grosso do Sul, Brasil, 2017.
At the end of the first round, the writing of 14 questions had to be modified: two in the "professional characterization" domain, one in "professional sensibility," eight in "professional knowledge/abilities," and three in "organization of the care network." The  domain "professional experience" was added, and nine new questions were inserted, two in "professional sensibility" and seven in "professional experience." The second version of the instrument had 43 items and five domains.
Seven questions had their wording changed, one in the "professional characterization" domain, three in "professional sensibility," two in "professional experience," and one in "professional knowledge." Seven questions were inserted, six in the "professional characterization" domain and one in the "professional knowledge/abilities" domain. This third version of the instrument was comprised of 50 items. The third round reached a consensus of 70% in 49 (98%) of the 50 items. Only one item, belonging to the "professional characterization" domain, had a lower level of agreement (67%).
In the fourth round, we asked for an evaluation of the only question that had not obtained a consensus, with the inclusion of information on the item's subject. The result was an agreement of 83%. The instrument was entitled Instrument for the Evaluation of Professional Assistance to People with Suicidal Behavior (IAAP-PCS). Its 50 items are divided into five domains: "professional characterization," "professional sensibility," "professional experience," "professional knowledge/abilities" and "organization of the care network." A five-point Likert-type scale was used for answering each item, except in the "professional characterization" domain. Data related to the maintenance, addition and modification of items are presented in the Box.
The characterization of the professionals who participated in the semantic analysis is shown in Table 2. It indicates that the instrument fulfilled its goals, receiving 93.6% "good" and "very good" evaluations.  The instrument's scale of responses was considered appropriate by 82% of the professionals, and 96% of the participants believed it was able to fulfill its evaluation purposes. According to 98%, the items are properly grouped and objectively formulated; all considered the instrument coherent. According to 88% of the professionals, items were easy to read and comprehend, as shown in the Figure. The general Cronbach's alpha of the questionnaire was 0.90 (excellent internal consistency), with 0.50 (poor internal consistency) in the "professional sensibility" domain, 0.90 (excellent internal consistency) in "professional experience," 0.82 (good internal consistency) in "professional knowledge/abilities," and 0.73 (acceptable internal consistency) in "organization of the care network." Table 3 presents the results of the internal consistency analysis of each domain's items.
The instrument innovates in establishing the evaluation of professional health care practice in the face of suicidal behavior, with easily understood items and a Likert-type scale of responses. The "professional characterization" domain covers the professional's training and qualification, time of service, and how frequently the professional encounters risk factors for suicide. The "professional sensibility" domain provides knowledge on how the professional perceives and understands the importance of the health service in the context of assisting people with suicidal behavior, constituting an important tool in a future process of sensitization. "Professional experience," a fundamental component, assesses how seasoned a professional is in certain activities, as higher levels of experience can benefit the provided assistance. Theoretical-practical knowledge makes it possible to provide assistance based on efficient health policies and reliable studies; this information is evaluated in the "professional knowledge/abilities" domain. Finally, for the provision of care, in addition to the professional's activity, a physical and medical structure is necessary, together with established work processes aimed at the formation of organized and efficient care networks. The "organization of care networks" domain aims to verify the local network's readiness to act in the prevention of suicide.

DISCUSSION
The IAAP-PCS instrument has the potential to enhance the integration of mental healthrelated care into the PHC system, since it allows assessing and identifying problematic aspects of care, as well as consolidated practices. It may also be useful for pointing out possible vulnerabilities and gaps in the country's mental health policy, in regards to the preparation of PHC professionals.
As in many countries, in Brazil this integration is occurring gradually, with a view to strengthening the primary care network and overcoming the country's history of unwarranted institutionalization practices targeting the mentally ill 13 . However, this process is not always accompanied by strategies for the evaluation of the provided services. This type of evaluation is essential for obtaining knowledge on past and current practices in the field of mental health, especially at this level of care 14 .
This instrument, designed to be applied to professionals working in PHC, can be an important tool in the development of best-practices for dealing with suicidal behavior, contributing to the field of epidemiology. The potential of the instrument to be used at both the individual and collective levels is in line with the stance advocated by several scholars, who point out that one of the main challenges is the inclusion of mental health care in the context of PHC, as part of a set of individual and collective, community-based actions [15][16][17] .
The lack of instruments to address suicidal behavior in PHC has led to the elaboration of the IAAP-PCS 18 . The instrument's domains are in accordance with World Health Organization policy and the protocols adopted by the Brazilian Ministry of Health. In the construction of the instrument, aspects related to suicide prevention strategies were taken into consideration -such as surveillance measures, identification of risk factors and vulnerable groups -as well as the importance of health professionals, especially those in PHC, given their direct contact with local health demands, which facilitates the identification of people with suicidal behavior. These strategies are also aligned with the proposals of scholars who research the thematic of care for people with suicidal behavior. Those scholars emphasize that mental health actions in the PHC should be directed to the promotion of health, prevention of aggravations and general treatment, considering the demands of the territory and emphasizing community participation in the processes of planning, operationalization and control [15][16][17] .
College-educated professionals were chosen to respond to the instrument due to their ability to identify risk factors, and also due to their involvement in follow-up 19 . It is important for practitioners to have the competence to understand their role in the process, and that they are prepared to act accordingly. Research indicates that professionals do not consider themselves prepared for this type of care and act primarily by referring patients to specialized services. In fact, most of the mental health actions identified consisted of team meetings, professional training and articulation between PHC and specialized services 14 .
The organization of the instrument consisted of domains that allow identifying potential areas of improvement in the perception and approach of the problem, making it possible to assess the preparation of PHC professionals in dealing with suicidal behavior, and to discuss their role.
The Delphi method is the most appropriate for instrument validation, since it seeks the consensus of professionals with great expertise in the field. Studies indicate that a 70% or higher level of consensus, the cut-off point used in this research, is enough to qualify the instrument [20][21][22][23][24] .
In the present study, after the Delphi rounds, semantic analysis was done with professionals who were part of the target population. In similar studies, this type of analysis has proven to be an important addition to instruments' validation process. One of these studies performed semantic analysis in order to verify if the items were intelligible to a meta-population of 15 technicians and nursing auxiliaries 21 . In another study, a scale was used to measure recovery in eight patients who had undergone intensive treatment. The patients themselves participated in the instrument's semantic analysis 25 . In the construction and validation of an instrument focused on good practices in the care of normal births, the semantic analysis was performed by eight health professionals, who again were part of the instrument's target population 26 . In our study, semantic analysis did not lead to changes in the instrument, since the respondent target population had a positive evaluation 25,26 .
As for internal consistency, the 0.90 value observed here is similar to results of other studies, which ranged from 0.87 to 0.91 27,28 .
In several studies, a consistency of 0.60 to 0.80 was also considered satisfactory 20,29 .
One limitation of the study concerns the semantic analysis step: it would be preferable for this stage to incorporate professionals of the entire health care network, so as to obtain an understanding on the system's mechanisms of communication, and on the interventions performed across the assistance network. We emphasize that, although the semantic analysis methodology adopted in this study recommends items to be verified from the perspective of strata of greater and lesser skill in the instrument's target population, no such analyzes were performed, since this was not the object of study.
The elaboration of the domains and its items started from an exhaustive search in the literature, in order to find scientific evidence about usual practices and what is necessary to make PHC assistance more qualified and adequate in the prevention of suicides. The employed methodological procedures (literature review, choice of the Delphi method to validate the instrument based on expert knowledge, semantic analysis for obtaining a broad intelligibility of the instrument, and analysis of internal consistency by Cronbach's alpha) were adequate for evaluating an instrument to assess the assistance provided to the people with suicidal behavior, qualifying the care network.
The IAAP-PCS can aid in epidemiological research and in the planning of actions to foster the practice of assessing the care provided to individuals with suicidal behavior, establishing agile and interconnected forms of care.