Mental Well-Being of Volunteer Rescue Workers After a Terrorist Attack: A Longitudinal Survey Study

287 görüntüleme

Sayı / Erken Görünüm
Sayı 1/2

Yıl / Cilt / Sayı
2022 / 1 / 2

Yazar/lar
Hans Scheers1, Anne-Catherine Vanhove2, Stijn Stroobants3, Emmy De Buck4, Karen Lauwers5, Philippe Vandekerckhove6

1 Centre for Evidence-Based Practice, Belgian Red Cross-Flanders Mechelen, Belgium; Department of Public Health and Primary Care, Leuven Institute for Healthcare Policy, KU Leuven, Leuven, Belgium

2 Centre for Evidence-Based Practice, Belgian Red Cross-Flanders, Mechelen, Belgium

3 Centre for Evidence-Based Practice, Belgian Red Cross-Flanders, Mechelen, Belgium

4 Centre for Evidence-Based Practice, Belgian Red Cross-Flanders, Mechelen, Belgium

5 Centre for Evidence-Based Practice, Belgian Red Cross-Flanders, Mechelen, Belgium

6 Centre for Evidence-Based Practice, Belgian Red Cross-Flanders, Mechelen, Belgium

Öz

This longitudinal study aims to explore the impact of the March 22, 2016, Brussels attacks on the mental health of the Belgian Red Cross-Flanders volunteers. Surveys were distributed to more than 350 volunteers who actively participated in response to the attacks and to more than 5000 other volunteers after one week and after 2, 6, 12, and 18 months. We examined posttraumatic stress symptoms in the volunteers using the Impact of Event Scale (IES) and their mental well-being using the Positive and Negative Affect Schedule (PANAS), both as a function of time after the attacks and deployment status in response to the attacks while adjusting for volunteering service and education level. The IES scores decreased over time, barring a slight increase on the anniversary of the attacks. However, positive and negative affects increased between the second and sixth month after the attacks and decreased subsequently. Besides higher negative affect scores in the latter group, scores did not significantly differ between volunteers actively or not actively involved in the response. Therefore, we conclude that volunteers of the Belgian Red Cross-Flanders on average had low posttraumatic stress symptoms and high mental well-being after the attacks in Brussels.

Tam Metin

People who experience traumatic events, such as natural and human-made disasters, including technological disasters and mass violence, are often negatively affected by these events (Norris et al., 2002). Because first responders are exposed to the direct consequences of disasters, increased exposure has been associated with a higher prevalence of posttraumatic stress disorder (PTSD); however, even first responders who only had contact with trauma survivors without experiencing the traumatic event can develop secondary traumatization (Greinacher, Derezza-Greeven, Herzog, & Nikendei, 2019; Neria, Nandi, & Galea, 2008). Most available studies on the mental health impact of disasters focus on natural and accidental disasters and the impact on direct victims or professional rescue workers (Bills et al., 2009; Brooks et al., 2015; Thormar et al., 2010). However, studies emphasizing the impact of terrorist attacks on volunteer rescue workers as a unique group are rare.

Nevertheless, numerous studies have been published concerning American Red Cross volunteers responding to the September 11, 2001, attacks. Elhai et al. (2006) reported that there was no overall significant increase in the mental health service use by 3055 American Red Cross disaster relief workers within the first year after the attack. However, several factors increased the likelihood of the use of services like no previous use of mental health services, being younger or divorced/widowed, and having higher PTSD intrusion or hyperarousal symptoms as assessed using the Impact of Event Scale-Revised (IES-R). In a second study using the same sample, directly or indirectly exposed volunteers, who went to the crash sites or had contact with other rescuers, survivors, and/or their families, did not exhibit significantly higher levels of posttraumatic stress or psychological distress compared with the non-exposed volunteers who were active in other roles (Long, Meyer, & Jacobs, 2007). In a different sample of 1500 Red Cross volunteers and employees, McCaslin et al. (2005) reported that negative life events in the year following the September 11, 2001, terrorist attacks were the only independent predictor of stress and symptoms of depression on the first anniversary of the attacks. Furthermore, Simons et al. (2005) reported that overall traumatic stress symptom scores were low, but higher hyperarousal and intrusion scores on the IES-R were associated with changes in alcohol consumption and greater hazardous consumption. A follow-up to this study found a consistent relation between the trait negative affect, coping motives, as well as alcohol abuse issues and that the younger participants were at the highest risk (Gaher, Simons, Jacobs, Meyer, & Johnson-Jimenez, 2006). However, a study focusing on 41 French Red Cross volunteers responding to the January 2015 terrorist attacks in Paris reported that nine interviewees had anxiety disorders, eleven had recurrent depressive episodes, and five met the criteria for alcohol abuse (Meudal, Vandentorren, Simeoni, & Denis, 2020).

These studies focused mainly on volunteers who actively participated in the response. Most studies warned that recollection bias might be distorting their results as they were based on a single survey conducted at some point from 6 to 12 months after the attacks. However, Long et al. cautioned that longitudinal patterns in posttraumatic stress symptoms and psychological distress in volunteers should be investigated, as research by Galea et al. in New York City residents suggested quick resolution of most probable PTSD symptoms within the first six months after the attacks (Galea et al., 2003; Long et al., 2007).

On March 22, 2016, terrorist attacks targeted two locations, Brussels Airport in Zaventem and Maalbeek metro station in Brussels city center, which resulted in 32 deaths and more than 300 injured. The Belgian Red Cross-Flanders immediately deployed both personnel and volunteered rescue workers to assist victims of the attacks. Volunteers were active in assisting the Civil Protection and Disaster Victim Identification teams, transporting injured victims to hospitals, setting up centers for noninjured victims, manning the call center of the crisis center, and coordinating and providing shelter as well as essential services to stranded travelers. Furthermore, the Belgian Red Cross-Flanders volunteers rendered psychosocial support to witnesses of the attacks as well as families of the fatalities. Consequently, more than 350 volunteers from three volunteering services (Relief Service [RS], Social Intervention Service [SIS], and Red Cross Youth [RCY]) provided disaster relief after the attacks. In addition, several volunteers were contacted and asked to be on standby (either on location or at home) but were not deployed, and several volunteers were contacted but were unable to respond (due to job or personal responsibilities). However, many volunteers were not contacted to help but were informed about the events through the media and were aware of the Belgian Red Cross-Flanders assisting in response.

After the attacks, RS, SIS, and RCY volunteers were surveyed several times by the professional core of the SIS to monitor the volunteers’ psychological well-being and posttraumatic stress symptoms concerning their actual deployment and tasks during the hours and days following the events and to provide and tailor specific resources and services that could aid in processing the events.

Purpose

The current study aimed to examine the evolution of the psychological well-being of the Belgian Red Cross-Flanders volunteers over the 18 months after the March 22, 2016, attacks in Brussels. Based on this survey, the study investigated the relation between their well-being and volunteering service and deployment status and postevent time using certain demographic characteristics.

The following hypotheses were investigated:

  1. The volunteers’ psychological well-being improved over postevent time.
  2. Actively deployed volunteers were—at least temporarily—more negatively affected because of increased exposure to the attacks and consequently, to potential traumatic events than those on standby (on location or at home) or those who were unavailable or not activated.

Methods

We used the guidelines of Strengthening the Reporting of Observational Studies in Epidemiology for reporting a prospective cohort study (Supplementary File 1; von Elm et al., 2007).

Study Population

Ten days after the terrorist attacks in Belgium, an invitation to participate in the survey was sent to all volunteers of the emergency RS at the time of events. Not only volunteers who were active after the attacks but also those who were on standby (on location or at home) or could not respond and those who were not activated, received an invite. However, owing to the inflow of new volunteers and outflow of volunteers leaving the Belgian Red Cross-Flanders, the number of invites sent varied slightly during the course of this study and was approximately 5900 at each five-time point; 5945 unique volunteers received the questionnaire at least once, and approximately 51% of them were male and 49% female. The average age of the volunteers was 40 years (Table 1). Volunteers within the emergency RS belonged to diverse but specific services, which included first responders (RS, n = approx. 5500), administrators and psychosocial support volunteers (SIS, n = approx. 275), and RCY (n = approx. 1000), with considerable overlap.

Procedure

Data collection. Multiple surveys were sent through email to all the Belgian Red Cross-Flanders volunteers active in the first responder and emergency RS. Emails were sent approximately 10 days (T1), 2 months (T2), 6 months (T3), 12 months (T4), and 18 months (T5) after the attacks that contained a link to the Questback website where the survey could be completed. Figure 1 illustrates the timeline of the surveys.

Figure 1. Timeline of the survey.

*At T2, deployment was defined differently, with only three categories: active, standby, not active

A reminder to fill out the survey was sent approximately two weeks after the first invitation, barring the first questionnaire for which no reminder was sent. For T2, both the original invite to the second questionnaire and its reminder were coupled with the invitation to the session to provide information and show appreciation to the volunteers. The response was relatively low since only those who wanted and could attend the session were presented with the survey. Therefore, a second reminder was sent approximately a month after the first reminder for this second survey.

Assessments of predictor variables. Participants were surveyed on their demographics and activities for the Belgian Red Cross-Flanders during and following the terrorist attacks. Demographic questions included age, gender, diploma, profession, postal code, as well as their usual volunteering service and the location they volunteered. Subsequently, the participants were asked to indicate whether they had been active, on standby on location, on standby at home, unable to respond to a call, or not activated after the attacks. Those who were active as the Belgian Red Cross-Flanders volunteers during and/or after the attacks were asked questions concerning their role assignment, the length of the assignment and breaks, contact with victims and/or their families, and the location of assignment. The participants who had been on standby on location were asked to describe the location. Questionnaires at T3 and T5 (6 and 18 months, respectively, after the events) comprised additional questions concerning the support offered by the Belgian Red Cross-Flanders to the volunteers after the attacks and whether these were used and appreciated by the volunteers. However, these data were not analyzed as they are beyond the scope of this study.

Measures of outcomes. The validated Dutch version of the IES was used to determine the frequency of intrusions associated with the trauma and avoidance behaviors (Horowitz, Wilner, & Alvarez, 1979; van der Ploeg, Mooren, Kleber, van der Velden, & Brom, 2004). The 15-item IES version instead of the 22-item IES-R (revised) was employed as the authors of the Dutch translations of both the IES and IES-R stated that the 15-item IES is the most suitable instrument for traumatic stress responses in general (Olde, Kleber, van der Hart, & Pop, 2006). The IES measures the frequency of each type of intrusion or avoidance behavior over the past seven days and is scored on a 4-point scale ranging from not at all (0 points), over rarely (1 point), sometimes (3 points), to often (5 points). The IES was administered during each of the five surveys that were sent. Summary scores for the intrusion as well as avoidance subscales were obtained by adding the scores for each of the seven and eight items, respectively, in each subscale. Thus, the intrusion ranged from 0 (no intrusion) to 35 (high intrusion) and avoidance behavior from 0 (no avoidance) to 40 points (high avoidance).

General mental well-being was measured using the Positive and Negative Affect Schedule (PANAS), a validated 20-item questionnaire that measures positive affect (PA) and negative affect (NA; Watson, Clark, & Tellegen, 1988). Ten PA items measure the extent to which a person experiences pleasurable engagement with the environment (e.g., feeling strong, enthusiastic, and alert.), whereas the 10 items measuring the NA reflect distress and unpleasurable engagement (e.g., feeling irritable, jittery, and guilty; Crawford & Henry, 2004; Watson et al., 1988). The participants scored each item on a 5-point scale from very slightly or not at all (1 point), a little (2 points), moderately (3 points), quite a bit (4 points) to very much (5 points).

In this study, the Dutch version (. Gedrag & Gezondheid: Tijdschrift voor Psychologie en Gezondheid, 34(2), 89–102.”]Engelen, De Peuter, Victoir, Van Diest, & Van Den Bergh, 2006) was used because it is the most commonly used translation in the Dutch-speaking region of Belgium, Flanders. The participants were asked to indicate for each item how they generally felt. The PANAS was first administered in the second survey and repeated in every subsequent survey. Based on the specific item scores, two summary scores were calculated: a PA and NA score, both ranging from 10 to 50 points.

Statistical Analyses

Outcome variables and their distribution. Visual inspection of the distribution of these outcome variables revealed that PA was skewed to the left, whereas the other outcomes were highly skewed to the right. Consequently, for intrusion, avoidance, and NA, the lowest possible value (i.e., 0, 0, or 10, respectively) was also the mode.

Visual inspection of quantile plots revealed that a negative binomial distribution best fitted data for intrusion, avoidance, and NA. These variables were further evaluated using generalized linear models (glm). However, none of the distributions tested (normal, lognormal, Poisson, negative binomial, and gamma) fitted the PA data well. Therefore, PA was transformed by subtracting PA from 60 and taking the log of the difference. This transformed variable followed a normal distribution and was further analyzed using linear models (lm).

Predictor variables. Four statistical models with the same set of predictor variables were constructed, one for each result (intrusion, avoidance, PA, and NA). To investigate the hypothesis that exposure to potentially traumatic events influenced posttraumatic stress, as measured using the IES, and affective state, as measured using the PANAS, the most general questionnaire item on the deployment was selected as a predictor variable. This categorical variable had five categories: active deployment, standby on location, standby at home, not contacted, and unavailable. Furthermore, the evolution of outcomes over time was measured by inserting questionnaire time points as a categorical predictor variable (called T1–T5) in the model.

Confounding variables. Simply put, a confounder is a variable associated with both the predictor and outcome variable, thereby causing a spurious association (Miettinen, 1974). Therefore, we evaluated possible confounding by four demographic variables: age, gender, education level (four categories: primary school, secondary school, bachelor’s degree, and postgraduate degree), and service (RS or SIS). RCY and mixed service profiles were excluded because these categories had few observations.

However, we investigated associations between these possible confounders, the four outcome variables (PA, NA, intrusion, and avoidance), and the main predictor variable (employment category). For pairs of continuous variables, we used correlation analysis combinations of a continuous and categorical variable, which were analyzed using ANOVA; for 2 × 2 tables of categorical variables, we used chi² tests of homogeneity. Regardless of the result of these pairwise comparisons, all demographic variables were inserted in the regression model as covariates.

Missing values. No formal imputation techniques were used for the missing values. Missingness was counteracted by copying invariable characteristics (i.e., deployment category and all covariates) within respondents with data on multiple time points. The randomness of missing values and representativity of the final dataset were checked by comparing characteristics between completers and noncompleters as well as between respondents and nonrespondents.

Regression analysis. We investigated associations between deployment, time, and the PANAS and IES outcomes by using mixed models, accounting for repeated measures within respondents. For intrusion, avoidance, and NA, we fitted a generalized linear mixed model (lmm) with a penalized quasilikelihood (glmmPQL) and negative binomial as the glm family. For PA, this was a lmm.

All descriptive statistics and analyses were performed in R (version 3.6.1), using the package stats (for basic statistics), lme4: linear mixed-effects models (function lmer), and MASS (functions glm.nb and glmmPQL). We performed crude analyses with time point (T1– T5), deployment category, and an interaction term as the only predictor variables as well as analyses adjusted for age, gender, education level, and volunteering service. In a sensitivity analysis, the postdisaster day was used as a continuous predictor variable instead of the categorical time point variable.

Findings

Descriptive Results

Response to the survey. Concerning questionnaires, 530, 913, 510, 499, and 443 were answered and returned after 10 days, 2 months, 6 months, 12 months, and 18 months, respectively (T1– T5) after the terrorist attacks. With 1600 unique responders out of 5945 volunteers invited, the overall response rate was 27%. Of these, 99 (1.7% of all receivers) responded to each of the five questionnaires. Conversely, 964 volunteers returned the questionnaire only once at any five-time points. Reported data for the main predictor variable (deployment category) and covariates (age, gender, service, and education level) at each time point are reported in Table 1. After the attacks, 34% of the responding volunteers were actively deployed, 7% were on standby at the location, and 10% were on standby at home. Of the overall volunteers, 39% had never been activated, and 11% were unavailable at the time of deployment; 71% of the responding volunteers belonged to the RS, 18% to the SIS, and 11% identified as “others.”

Table 1

Response Rates and Predictor Variables at Each Time Point

Total

contacted*

Total responded$ Time point 1$ Time point 2$ Time point 3$ Time point 4$ Time point 5$
n % n % n % n % n % n % n %
Contacted 5945 100 / 5878 100 5882 100 5736 100 5880 100 5882 100
Responded§ / 1190 20 530 9 187 3 510 9 499 8 443 8
Gender£ 5130 86 1180 99 528 100 184 98 509 100 497 100 441 100
Male 2596 51 674 57 293 55 106 58 300 59 283 57 260 59
Female 2534 49 506 43 235 45 78 42 209 41 214 43 181 41
Service£ 5727 96 524 44 524 99 93 50 180 35 194 39 177 40
RS 5483 96 371 71 371 71 45 48 109 61 116 60 105 59
SIS 243 4 94 18 94 18 34 37 51 28 56 29 54 31
Other 1 0 57 11 57 11 14 15 20 11 22 11 18 10
Education level£ not available 528 44 528 100 93 50 181 35 195 39 178 40
Primary school 29 5 29 5 5 5 13 7 11 6 7 4
Secondary school 199 38 199 38 32 34 57 31 66 34 56 31
Bachelor 182 34 182 34 28 30 57 31 62 32 55 31
Master 118 22 118 22 28 30 54 30 56 29 60 34
Deployment£ 5945 100 1023 86 523 99 187 100 386 76 474 95 428 97
Active 361 6 346 34 219 42 129 69 178 46 197 42 187 44
Standby on location 67 7 26 5 18 10 30 8 27 6 25 6
Standby at home 5584 94 102 10 50 10 4 2 29 8 53 11 39 9
Not activated 396 39 203 39 10 5 100 26 164 35 150 35
Unavailable 112 11 25 5 26 14 49 13 33 7 27 6
Age£† 5103 86 920 77 430 81 184 98 448 88 436 87 368 83
Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD
Years 39.7 13.9 39.7 13.9 39.1 13.3 40.9 13.0 40.0 13.9 39.9 13.8 40.1 13.3

RS: Relief Service; SIS: Social Intervention Service; SD: standard deviation

*As retrieved from the central database of Belgian Red Cross-Flanders volunteers

$As reported by the respondents to the questionnaire

  • Relative to the cell above, thus representing the response rate

£Relative to the number in the row “Responded”, thus representing the rate of non-missing values; All other percentages are relative to the category total, thus representing the distribution of categories, not counting missing values

Age on 22nd March 2016

With only 187 responses after a month, the response to the second questionnaire, including the reminder, was low at first. This was a result of the survey being associated with an invitation to an information session at the Belgian Red Cross-Flanders headquarters. Only respondents who were willing and able to attend filled the form after which they reached the survey. Therefore, a second reminder was sent on June 30, 2016. The response to the second reminder was unexpectedly large, with 726 additional responses. This response was most likely due to the terrorist attacks on Atatürk Airport in Istanbul, Turkey, which occurred only two days before the response. We found that NA and avoidance scores were higher in respondents of the second reminder (after the Istanbul attacks) than in those who returned the second questionnaire after it was initially sent or after the first reminder (t-tests, p < .001 and p = .008, respectively). PA and intrusion scores did not differ between the first and second waves (both p > .50).

Consequently, we removed responses from the second reminder from further analyses to reduce bias. Thus, the total number of unique responders decreased from 1600 to 1190.

Missing data. Missing data occurred because of limitations in the stage of questionnaire development and incomplete response. First, the design of the questionnaires unintentionally induced missing data. The date of birth (necessary to calculate age on the day of the attacks) was included in the optional part with identification data (name, surname, and address) and therefore, not disclosed by approximately 25% of the respondents who wanted to maintain anonymity.

Second, the questionnaire at timepoint 2 could only be reached by those indicating that they wanted to attend an information session (see above).

Third, several variables of interest were excluded from each of the five questionnaires (Figure 1). The deployment category was omitted from the third questionnaire and defined differently in the second questionnaire (with only three categories: active, on standby, and not active). The covariates education level and volunteering service only were included in the first questionnaire, and the respondent’s gender was not questioned in the second questionnaire. The IES outcomes were included at each time point, but the outcomes of the PANAS were included only in the second questionnaire. We could reduce the number of missing values by copying those invariable characteristics (all the above, except the PANAS), which were answered at least once by nonanonymous respondents.

The overall response rate was low (27%, see above), with high dropout and (re-)entry rates, leading to only a few respondents with full data. In addition, many respondents did not reach the end of the questionnaire or left one or more items unanswered.

Representativity of the survey. The 1190 responders included in the analyses were a representative subset of the entire population of the RS and SIS volunteers in terms of age (responders 39.9 ± 13.6 years; nonresponders 39.6 ± 14.0 years; t-test: p = .44) but not for gender, service, and the deployment category. Of the responders, 57% were males, and they made up 51% of the volunteers (chi² test: p < .0001). Of the SIS volunteers, 64% responded to the questionnaire, in stark contrast to only 19% of the RS volunteers who returned the questionnaire (chi² test: p < .0001). Among volunteers who had been activated after the attacks, the response rate was 63%, whereas it was only 18% among those who had not been deployed (chi² test: p < .0001). For the education level, representativity could not be tested because this self-reported variable was only available for responders to the questionnaire.

Outcome scores. In all the questionnaires, the median overall PA score was 38 points (interquartile range [IQR] 34–42) and the median NA score was 16 points (IQR 12–21). Both scores ranged from 10 to 50 points. Concerning the IES, the median intrusion score was 3 (IQR 0–8) and ranged from 0 to 35 points, whereas the median avoidance score was 1 (IQR 0–6; range 0–40).

Results of Analyses

Confounding

Gender, age, education level, and volunteering service were statistically significantly associated with one or more result variables (PA, NA, intrusion, and avoidance). Moreover, the education level and volunteering service were associated with the deployment category (Supplementary Table 1). Relatively more volunteers from the SIS were active after the terrorist attacks than first responders from the RS (p < .0001). Because the psychosocial volunteers of the SIS at that time were required to hold at least a bachelor’s degree in a psychosocial field of study, relatively more volunteers with a bachelor’s or master’s degree were more active than volunteers with primary or secondary school degrees (p < .0001). Hence, as they were significantly associated with both outcome and predictor variables, the education level and volunteering service were potential confounders of the association between deployment and indicators of well-being.

PANAS Analyses

Table 2 presents the results of the crude and adjusted analyses of the associations between the deployment category, time point, and the PANAS outcomes. Time was analyzed as a categorical variable (the different time points of the surveys from T2 to T5). An increase in both PA and NA from T2 to T3, followed by a slight decrease (PA crude: p < .001; PA adjusted: p < .001; NA crude: p < .001; NA adjusted: p < .001) was observed. This pattern was confirmed in the adjusted analysis using a postdisaster day as a continuous time variable predictor variable: we discovered a quadratic association with PA and NA, which increased to a maximum on Day 384 (p = .035) and Day 411 (p = .031), respectively, and gradually decreased thereafter (Supplementary Table 2). Quadratic terms in the unadjusted analyses of PA and NA were even more pronounced (both p < .001). Parameter estimates for the deployment categories were nearly identical to those obtained using the categorical time variable.

Table 2

Results of Regression Analyses for the PANAS

Outcome Predictor variables Crude analysis Adjusted analysis*
    Difference 95% CI p and pattern$ n Difference 95% CI p and pattern$ n§
      LL UL     LL UL  
PA Deployment .006 1370 .13 529
Active reference B reference A
Standby on location -2.2 -4.5 0.0 A,B 3.9 -0.7 7.8 A
Standby at home -1.1 -3.0 0.7 A,B 0.5 -3.9 4.4 A
Unavailable -1.9 -3.9 -0.1 A,B -3.6 -9.7 1.5 A
Not activated -2.1 -3.3 -0.9 A -1.6 -4.5 1.0 A
Time point < .001 .001
T2 reference A reference A
T3 3.1 2.3 3.9 C 3.3 1.8 4.7 B
T4 2.3 1.5 3.2 B 2.0 0.4 3.5 B
T5 2.3 1.5 3.2 B 2.3 0.8 3.8 B
NA Deployment < .001 1376 < .001 530
Active reference A reference A
Standby on location 0.1 -1.3 1.7 A,B -2.0 -4.3 0.9 A
Standby at home 2.0 0.7 3.5 B 6.0 2.8 9.8 B
Unavailable 1.7 0.4 3.1 B 6.0 2.3 10.6 B
Not activated 2.1 1.3 3.1 B 2.2 0.5 4.2 B
Timepoint < .001 < .001
T2 reference A reference A
T3 1.5 0.8 2.4 B 2.3 1.1 3.7 B
T4 1.4 0.6 2.2 B 1.6 0.4 2.9 B
T5 0.9 0.1 1.7 B 1.7 0.4 3.0 B

Mixed regression models showing differences in parameter estimates between deployment categories and timepoints for the outcomes PA and NA

PANAS: Positive and Negative Affects Scale; PA: Positive Affects; NA: Negative Affects; CI: confidence interval; LL: lower limit; UL: upper limit

*Adjusted for age, gender, education level and volunteering service

$p for overall effect of the categorical variable; groups with the same letter are not statistically different from each other in a Tukey post-hoc analysis (p > .05)

  • N is lower in the adjusted analysis because of missing values for the covariates

In the crude analyses, we observed a higher PA score in volunteers that were actively deployed after the attacks than in those who were not activated (2.1 points higher in active, 95% confidence interval (CI) [0.9; 3.3], p = .006), with no differences between the other deployment categories. Actively deployed volunteers had lower NA scores than those who were on standby at home (2.0 points lower in active, 95% CI [0.7;3.5]), unavailable (1.7 points lower, 95% CI [0.4;3.1]), or not activated (2.1 points lower, 95% CI [1.3;3.1]). This difference remained in the adjusted analysis for NA (p < .001; with additionally significantly lower scores for standby on location than standby at home, unavailable, or not activated), but disappeared in the adjusted analysis for PA (p = .13).

Interaction terms between time and deployment were not significant; therefore, they were omitted from all analyses.

IES Analyses

The results of the crude and adjusted analyses of the associations between the deployment category, time point, and the IES outcomes are presented in Table 3. Time was analyzed as a categorical variable (the different time points of the surveys). A significant decrease in both intrusion and avoidance over time (overall p < .001 for the crude and adjusted analyses) was observed, with a notable increase at T4, exactly one year after the attacks (values for intrusion and avoidance at T4 significantly lower than that at T1 but higher than that at T2, T3, and T5). This pattern was roughly confirmed in the analysis with a postdisaster day as a continuous time and predictor variable, where a linear decrease was found in intrusion and avoidance over time (all p < .001 for the crude and adjusted analyses of intrusion and avoidance), with no indication of a quadratic association (all p > .31 for the crude and adjusted analyses of intrusion and avoidance; Supplementary Table 3).

Table 3

Results of Regression Analyses for the IES

Outcome Predictor variables Crude analysis Adjusted analysis*
    Difference 95% CI p and pattern$ n Difference 95% CI p and pattern$ n§
      LL UL     LL UL  
Intrusion Deployment .56 1742 .98 827
Active reference A reference A
Standby on location -0.6 -2.2 1.6 A -0.7 -4.5 5.8 A
Standby at home 0.6 -1.0 2.9 A 1.2 -3.0 7.9 A
Unavailable -0.4 -1.9 1.6 A -0.6 -4.9 7.4 A
Not activated 0.9 -0.3 2.4 A 0.8 -2.0 4.7 A
Time point < .0001 .<0.0001
T1 reference A reference A
T2 -3.3 -3.7 -2.7 C -5.9 -6.9 -4.7 C
T3 -3.7 -4.0 -3.3 C -6.8 -7.4 -6.1 C
T4 -2.2 -2.6 -1.6 B -3.1 -4.3 -1.7 B
T5 -4.2 -4.4 -4.0 D -7.1 -7.6 -6.4 C
Avoidance Deployment < .001 1733 .19 820
Active reference A reference A
Standby on location 0.5 -0.7 2.4 A,B -1.0 -3.4 3.6 A
Standby at home 1.7 0.3 3.9 B 2.3 -1.2 8.5 A
Unavailable 1.3 -0.1 3.4 A,B 2.0 -2.1 10.8 A
Not activated 2.3 1.2 3.8 B 3.0 0.2 7.0 A
Timepoint < .001 < .001
T1 reference A reference A
T2 -1.8 -2.0 -1.6 C -3.7 -4.3 -2.8 C
T3 -1.8 -1.9 -1.6 C -4.1 -4.5 -3.6 C
T4 -1.4 -1.6 -1.1 B -2.5 -3.2 -1.6 B
T5 -2.2 -2.3 -2.1 D -4.7 -5.0 -4.3 D

Mixed regression models showing differences in parameter estimates between deployment categories and timepoints for the outcomes intrusion and avoidance

IES: Impact of Event Scale; CI: confidence interval; LL: lower limit; UL: upper limit

*Adjusted for age, gender, education level and volunteering service

$p for overall effect of the categorical variable; groups with the same letter are not statistically different from each other (p > .05) in a Tukey post-hoc analysis

  • N is lower in the adjusted analysis because of missing values for the covariates

In the crude analyses, we observed a lower avoidance score in active volunteers than in volunteers that were not activated (2.3 points lower in active, 95% CI [1.2;3.8]) or standby at home (1.7 points lower, 95%CI [0.3;3.9]; p < .001). However, this difference was not observed for intrusion (crude: p = .56; adjusted: p = .98) and it disappeared in the adjusted analysis for avoidance (p = .19).

Again, interaction terms between time and deployment were not significant; therefore, they were omitted from all analyses.

Discussion

Main Findings

Generally, scores on the IES were extremely low and way below the cutoff of 19 or 25 points that typically are used to indicate the moderate or severe impact of events, respectively (Epstein, Fullerton, & Ursano, 1998; Witteveen et al., 2007). This finding corresponds to other studies on the Red Cross and Red Crescent volunteers responding to terrorist attacks, which found low levels of posttraumatic stress symptoms, mental health service use, and alcohol abuse issues (Elhai et al., 2006; Long et al., 2007; McCaslin et al., 2005; Simons et al., 2005).

Confirming our expectations, which is consistent with earlier research (Galea et al., 2003), a decrease in intrusion and avoidance (IES) occurred over time. There was a slight increase in both avoidance and intrusion in Questionnaire 4, which was expected as this questionnaire was sent right after the first anniversary of the terrorist attacks. This anniversary, which was covered extensively by the media, explicitly reminded many volunteers about the traumatic events of the previous year, which led to a slight increase in symptoms. Moreover, this anniversary reaction has been described by rescue workers responding to the September 11, 2001, terrorist attacks in New York (Daly et al., 2008).

Both positive and negative affects increased between the second and sixth month after the attacks and decreased subsequently. However, the layout of the PANAS questionnaire differed between T2 (two months) and the other time points, which might have slightly affected the score. This study’s limitations are discussed below in more detail.

The second hypothesis concerning active volunteers being more negatively affected owing to increased exposure to the attacks was not confirmed as no statistical differences in mean scores were identified between the groups for avoidance or intrusion (IES) or PA (PANAS) in the adjusted analyses. Conversely, NA (PANAS) was higher in the volunteers who were unavailable, on standby at home, or not activated compared with those who were active or on standby on the location.

One possible reason why active volunteers are not more negatively affected could be the support provided by a volunteer organization, such as the Belgian Red Cross-Flanders. Several support structures, which are resilience factors and prevent the development of secondary traumatization (Greinacher et al., 2019), were in place after the occurrence of the disasters. Another explanation for the higher NA scores in volunteers who were not actively involved in the response is the possibility that they were highly motivated to assist but somewhat frustrated at being unable to do so. However, the use of email surveys could also have affected results as they might not be representative of the entire population of the volunteers. An alternative explanation could be that more negatively affected active volunteers did not want to be confronted with the questionnaire; therefore, they did not fill it. However, the nonactive volunteers feeling less involved in the response might have been more motivated to fill the survey as a way to contribute.

Education as a Confounder

The education level and volunteering service were identified as potential confounders, which were associated with both outcomes (PANAS and IES) and the predictor variable deployment category (active, standby on location or at home, unavailable, or not activated). The latter association is not surprising as relatively more volunteers from the SIS were deployed. However, as this department is made up of volunteer administrators and psychosocial volunteers, this group was required to obtain at least a bachelor’s degree in a psychosocial area of study, and those with higher education levels belonging to the SIS were deployed more.

Furthermore, volunteers with higher education levels belonging to the SIS were found to have better mental health outcomes regardless of their deployment status. In the actively deployed volunteers, this difference could be due to different tasks, experiences, as well as exposure during the response. However, those who were educated and trained in psychosocial support might be more resilient during and after the response to stressful situations in general. Among the Belgian Red Cross-Flanders volunteers, intrusion scores on the IES were not different between those with lower or higher education levels, but those with bachelor’s and master’s degrees had lower avoidance scores than those with primary or secondary education degrees.

Volunteers with higher education levels as first responders exhibit less posttraumatic stress after responding to a disaster (Epstein et al., 1998; Motreff et al., 2020; Noda, Asano, Shimizu, & Hirano, 2018). However, other studies did not confirm these observations and found no association (Lee et al., 2017; Ma, Chang, Wu, & Lin, 2020; Thormar et al., 2014; Witteveen et al., 2007). Similar findings are also described in the general population’s response to disaster with either a positive or nonsignificant association between those with higher education and less posttraumatic stress symptoms (Tang, Deng, Glik, Dong, & Zhang, 2017).

Moreover, we found that volunteers with higher education levels had lower NA scores, whereas their PA scores did not differ from those volunteers with lower education levels. While PA and NA, measured using the PANAS, are not often used in posttraumatic disaster research, the correlation between PA or NA and education has been studied in everyday life. Möwisch et al. (2021), for instance, found similar lower NA and equal PA scores in volunteers with higher education levels compared with those with lower education levels, but several other studies have identified higher PA scores in volunteers with higher education levels; associations between NA scores and education can be positive, absent, or even negative (Collins, Sarkisian, & Winner, 2009; Crawford & Henry, 2004; Nikolaev, 2018).

Strengths and Limitations

To the best of our knowledge, this study is the first longitudinal study following the mental well-being of volunteer first responders to a terrorist attack. All the Belgian Red Cross-Flanders volunteers of the RS, SIS, and RCY services were surveyed for the first time within a week of the March 22, 2016, attacks in Brussels and received a total of five questionnaires until September 2017. The importance of performing longitudinal studies on volunteers responding to terrorist attacks has already been highlighted (Long et al., 2007; McCaslin et al., 2005). Our observation that the IES scores demonstrated a small peak on the first anniversary supports the potential limitation hypothesized by McCaslin et al. who conducted a single survey on the first anniversary of the 9/11 attacks, stating that responders’ increased thoughts of those attacks might influence their IES-R measurement.

We consider our focus on both activated and nonactivated volunteers’ additional strength. While research on psychological distress and well-being in disaster rescue workers responding to terrorist attacks usually only focuses on those actively involved in the immediate or long-term response (Elhai et al., 2006; Long et al., 2007; McCaslin et al., 2005; Meudal et al., 2020; Motreff et al., 2020), this study surveyed all volunteers belonging to the respective operational services of the Belgian Red Cross-Flanders, both those who were active in the response and those on standby (on location or at home), unavailable, or not activated. Therefore, this study is the first to show that volunteers actively involved in the response on standby or location are not necessarily more negatively affected than their nonactive counterparts.

The usual limitations of longitudinal survey study design apply to this study. Of approximately 6000 volunteers that were surveyed through email at each five-time point, 1600 (27%) of them filled at least a survey, with less than 100 filling all five surveys. This could introduce possible nonresponse bias and, even after removing the responders to the second reminder of the second questionnaire, it remains uncertain if the study sample is representative of the entire volunteer population of the Belgian Red Cross-Flanders concerning the impact of the attacks on their mental well-being.

Certainly, a higher proportion of actively deployed volunteers (63%) than nondeployed volunteers (18%) responded to the survey. This ratio was expected and did not necessarily lead to bias. However, as discussed above, more negatively affected active volunteers might be less represented as they might not have wanted to participate in the survey. Conversely, nonactive volunteers who felt frustrated because of their inability to help might have been more motivated to fill the questionnaire. Moreover, a significantly higher proportion of the SIS volunteers compared with the RS volunteers filled the survey, and males were presented as well. Therefore, it remains uncertain whether the response in this sample represents the actual mental well-being of the RS and SIS volunteers as a whole.

Furthermore, several other potential limitations need to be stated. First, the survey was developed to monitor the mental health of volunteers after the attacks and to assess which resources provided by the Belgian Red Cross-Flanders helped the volunteers with processing these events. Moreover, the responses were used to offer additional support to volunteers who did not cope as well as their peers. This contact may have influenced their later participation and responses in the surveys, but reflects standard support practice by the Belgian Red Cross-Flanders. Second, the initial invitation to the second survey and first reminder linked to an invitation to an information session. Only those who were willing and able to attend the information session responded to the invitation and accessed the questionnaire, which might have led to a different sample filling the survey than if the survey had been sent separately. This limitation is supported by the considerably low response rate of only 3% compared with the usual 8%–9%. However, the second reminder for this second survey coincided with the Istanbul Atatürk airport attacks, which led to a large increase in responses. This second reminder resulted in 726 responses, more than in any other survey. Unfortunately, these later responses were markedly different from the responses received before the reminder was sent. Late responders had higher NA and avoidance scores than those who responded to the first invitation to the questionnaire, similar to the effect observed in the fourth survey that was administered on the first anniversary of the Brussels attacks. The Atatürk airport and anniversary of the Brussels attack triggered memories of the events of March 22, 2016, resulting in reduced mental well-being during that period. While neither of the samples of respondents to this second survey was free of bias, we chose to remove the responses after the second reminder from the analysis.

In addition, involuntary inconsistencies that were introduced during the survey designs resulted in discrepancies between the surveys at different time points; therefore, ultimately, some variables were excluded from the analysis. First, the PANAS and IES were not always incorporated in the survey at the same location or in the same order. At T2 and T3, they were the final items of the survey, and the IES was filled first, followed by the PANAS. However, at T4 and T5, they were the first items that the respondents filled, and the PANAS was filled first, followed by the IES. Second, the PANAS was surveyed using a word rating scale at T2 but with some rating scales using the legend at T3, T4, and T5. These small differences might have affected the scores of the PANAS and IES and should be avoided in future longitudinal surveys. Third, some exposure-related questions were asked only at a few or even only one-time points and could not be linked across questionnaires, which would have resulted in a lower number of questionnaires relevant for analysis. Other questions were inconsistently surveyed with slightly different response options. Finally, several questions were considered slightly too subjective (e.g., “Were you exposed to something you would rather not have seen?”) to be reliably linked to actual exposure to the disaster. Therefore, we had to select the most general question on deployment status as a predictor variable rather than questions concerning contact with victims and/or families, the length of the assignment, assignment location, etc. Finally, both the posttraumatic stress symptoms and affect of the volunteers were measured by self-assessment questionnaires, and for obvious reasons, no baseline, pre-attacks measurement could be conducted.

Conclusion

This study provides evidence concerning the fact that volunteers of the Belgian Red Cross-Flanders had low posttraumatic stress symptom scores (rated with the IES) after the March 2016 attacks in Brussels. Barring a temporary increase in both intrusion and avoidance on the first anniversary of the attacks, the IES scores further decreased over time. Moreover, posttraumatic stress symptoms and PA did not significantly differ among volunteers who were actively deployed, on standby (on location or at home), unavailable, or not activated. Volunteers who were present at the location of the attacks (either active or on standby) had lower NA scores (i.e., better mental health) than those who were not.

Further longitudinal studies should estimate volunteers’ degree of exposure to potentially traumatic events with more focused questions to allow for a better analysis of its association with their mental health. In addition, the relation between posttraumatic stress symptoms and the availability as well as the need for and use of mental health support services provided by and outside the volunteer organization should be further explored.

Kaynakça

Bills, C. B., Dodson, N., Stellman, J. M., Southwick, S., Sharma, V., Herbert, R., Moline, J. M., & Katz, C. L. (2009). Stories behind the symptoms: A qualitative analysis of the narratives of 9/11 rescue and recovery workers. Psychiatric Quarterly, 80(3), 173–189. https://doi.org/10.1007/s11126-009-9105-7

Brooks, S. K., Dunn, R., Sage, C. A., Amlot, R., Greenberg, N., & Rubin, G. J. (2015). Risk and resilience factors affecting the psychological wellbeing of individuals deployed in humanitarian relief roles after a disaster. Journal of Mental Health, 24(6), 385–413. https://doi.org/10.3109/09638237.2015.1057334

Collins, A. L., Sarkisian, N., & Winner, E. (2009). Flow and happiness in later life: An Investigation into the role of daily and weekly flow experiences. Journal of Happiness Studies, 10(6), 703–719. https://doi.org/10.1007/s10902-008-9116-3

Crawford, J. R., & Henry, J. D. (2004). The positive and negative affect schedule (PANAS): Construct validity, measurement properties and normative data in a large non-clinical sample. British Journal of Clinical Psychology, 43(Pt 3), 245–265. https://doi.org/10.1348/0144665031752934

Daly, E. S., Gulliver, S. B., Zimering, R. T., Knight, J., Kamholz, B. W., & Morissette, S. B. (2008). Disaster mental health workers responding to Ground Zero: One year later. Journal of Traumatic Stress, 21(2), 227–230. https://doi.org/10.1002/jts.20311

Elhai, J. D., Jacobs, G. A., Kashdan, T. B., DeJong, G. L., Meyer, D. L., & Frueh, B. C. (2006). Mental health service use among American Red Cross disaster workers responding to the September 11, 2001 U.S. terrorist attacks. Psychiatry Research, 143(1), 29–34. https://doi.org/10.1016/j.psychres.2005.10.004

Engelen, U., De Peuter, S., Victoir, A., Van Diest, I., & Van Den Bergh, O. (2006). Verdere validering van de Positive and Negative Affect Schedule (PANAS) en vergelijking van twee Nederlandstalige versies. [Further validation of the Positive and Negative Affect Schedule (PANAS) and comparison of two Dutch versions.]. Gedrag & Gezondheid: Tijdschrift voor Psychologie en Gezondheid, 34(2), 89–102.

Epstein, R. S., Fullerton, C. S., & Ursano, R. J. (1998). Posttraumatic stress disorder following an air disaster: A prospective study. American Journal of Psychiatry, 155(7), 934–938. https://doi.org/10.1176/ajp.155.7.934

Gaher, R. M., Simons, J. S., Jacobs, G. A., Meyer, D., & Johnson-Jimenez, E. (2006). Coping motives and trait negative affect: testing mediation and moderation models of alcohol problems among American Red Cross disaster workers who responded to the September 11,2001 terrorist attacks. Addictive Behaviors, 31(8), 1319–1330. https://doi.org/10.1016/j.addbeh.2005.10.006

Galea, S., Vlahov, D., Resnick, H., Ahern, J., Susser, E., Gold, J., Bucuvalas, M., & Kilpatrick, D. (2003). Trends of probable post-traumatic stress disorder in New York City after the September 11 terrorist attacks. American Journal of Epidemiology, 158(6), 514–524. https://doi.org/10.1093/aje/kwg187

Greinacher, A., Derezza-Greeven, C., Herzog, W., & Nikendei, C. (2019). Secondary traumatization in first responders: A systematic review. European Journal of Psychotraumatology, 10(1), 1562840. https://doi.org/10.1080/20008198.2018.1562840

Horowitz, M., Wilner, N., & Alvarez, W. (1979). Impact of Event Scale: A measure of subjective stress. Psychosomatic Medicine, 41(3), 209–218. https://doi.org/10.1097/00006842-197905000-00004

Lee, J. Y., Kim, S. W., Bae, K. Y., Kim, J. M., Shin, I. S., & Yoon, J. S. (2017). Factors associated with posttraumatic stress disorder symptoms among community volunteers during the Sewol ferry disaster in Korea. Comprehensive Psychiatry, 77, 38–44. https://doi.org/10.1016/j.comppsych.2017.05.007

Long, M. E., Meyer, D. L., & Jacobs, G. A. (2007). Psychological distress among American Red Cross disaster workers responding to the terrorist attacks of September 11, 2001. Psychiatry Research, 149(1–3), 303–308. https://doi.org/10.1016/j.psychres.2006.05.022

Ma, I. C., Chang, W. H., Wu, C. L., & Lin, C. H. (2020). Risks of post-traumatic stress disorder among emergency medical technicians who responded to the 2016 Taiwan earthquake. Journal of the Formosan Medical Association, 119(9), 1360–1371. https://doi.org/10.1016/j.jfma.2019.11.021

McCaslin, S. E., Jacobs, G. A., Metzler, T. J., & Marmar, C. R. (2005). How does negative life change following disaster response impact distress among red cross responders? Professional Psychology-Research and Practice, 36(3), 246–253. https://doi.org/10.1037/0735-7028.36.3.246

Meudal, J., Vandentorren, S., Simeoni, L., & Denis, C. (2020). French Red Cross volunteer rescue workers: Psychological characteristics and healthcare support after the january 2015 terrorist attacks in Paris. The Journal of Nervous and Mental Disease, 208(5), 413–417. https://doi.org/10.1097/nmd.0000000000001143

Miettinen, O. (1974). Confounding and effect-modification. American Journal of Epidemiology, 100(5), 350–353. https://doi.org/10.1093/oxfordjournals.aje.a112044

Motreff, Y., Baubet, T., Pirard, P., Rabet, G., Petitclerc, M., Stene, L. E., Vuillermoz, C., Chauvin, P., & Vandentorren, S. (2020). Factors associated with PTSD and partial PTSD among first responders following the Paris terror attacks in November 2015. Journal of Psychiatric Research, 121, 143–150. https://doi.org/10.1016/j.jpsychires.2019.11.018

Mowisch, D., Brose, A., & Schmiedek, F. (2021). Do higher educated people feel better in everyday life? Insights from a day reconstruction method study. Social Indicators Research, 153, 227–250. https://doi.org/10.1007/s11205-020-02472-y

Neria, Y., Nandi, A., & Galea, S. (2008). Post-traumatic stress disorder following disasters: A systematic review. Psychological Medicine, 38(4), 467–480. https://doi.org/10.1017/s0033291707001353

Nikolaev, B. (2018). Does higher education increase hedonic and eudaimonic happiness? Journal of Happiness Studies, 19(2), 483–504. https://doi.org/10.1007/s10902-016-9833-y

Noda, Y., Asano, K., Shimizu, E., & Hirano, Y. (2018). Assessing subgroup differences in posttraumatic stress disorder among rescue workers in Japan with the impact of event scale-revised. Disaster Medicine and Public Health Preparedness, 1–10. https://doi.org/10.1017/dmp.2018.62

Norris, F. H., Friedman, M. J., Watson, P. J., Byrne, C. M., Diaz, E., & Kaniasty, K. (2002). 60,000 disaster victims speak: Part I. An empirical review of the empirical literature, 1981-2001. Psychiatry, 65(3), 207–239. https://doi.org/10.1521/psyc.65.3.207.20173

Olde, E., Kleber, R. J., van der Hart, O., & Pop, V. J. M. (2006). Childbirth and posttraumatic stress responses: A validation study of the Dutch Impact of Event Scale-Revised. European Journal of Psychological Assessment, 22(4), 259–267. https://doi.org/10.1027/1015-5759.22.4.259

Simons, J. S., Gaher, R. M., Jacobs, G. A., Meyer, D., & Johnson-Jimenez, E. (2005). Associations between alcohol use and PTSD symptoms among American Red Cross disaster relief workers responding to the 9/11/2001 attacks. The American Journal of Drug and Alcohol Abuse, 31(2), 285–304.

Tang, B., Deng, Q., Glik, D., Dong, J., & Zhang, L. (2017). A Meta-Analysis of Risk Factors for Post-Traumatic Stress Disorder (PTSD) in adults and children after earthquakes. International Journal of Environmental Research and Public Health, 14(12). https://doi.org/10.3390/ijerph14121537

Thormar, S. B., Gersons, B. P., Juen, B., Djakababa, M. N., Karlsson, T., & Olff, M. (2014). The impact of disaster work on community volunteers: The role of peri-traumatic distress, level of personal affectedness, sleep quality and resource loss, on post-traumatic stress disorder symptoms and subjective health. Journal of Anxiety Disorders, 28(8), 971–977. https://doi.org/10.1016/j.janxdis.2014.10.006

Thormar, S. B., Gersons, B. P., Juen, B., Marschang, A., Djakababa, M. N., & Olff, M. (2010). The mental health impact of volunteering in a disaster setting: A review. The Journal of Nervous and Mental Disease, 198(8), 529–538. https://doi.org/10.1097/NMD.0b013e3181ea1fa9

van der Ploeg, E., Mooren, T. T., Kleber, R. J., van der Velden, P. G., & Brom, D. (2004). Construct validation of the Dutch version of the impact of event scale. Psychological Assessment, 16(1), 16–26. https://doi.org/10.1037/1040-3590.16.1.16

von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., & Vandenbroucke, J. P. (2007). The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies. PLoS Med, 4(10), e296. https://doi.org/10.1371/journal.pmed.0040296

Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54(6), 1063–1070. https://doi.org/10.1037//0022-3514.54.6.1063

Witteveen, A. B., Bramsen, I., Twisk, J. W., Huizink, A. C., Slottje, P., Smid, T., & Van Der Ploeg, H. M. (2007). Psychological distress of rescue workers eight and one-half years after professional involvement in the Amsterdam air disaster. The Journal of Nervous and Mental Disease, 195(1), 31–40. https://doi.org/10.1097/01.nmd.0000252010.19753.19

Sitemizden en iyi şekilde faydalanabilmeniz için çerezleri kullanmaktayız.    Daha Fazla Bilgi