KSOEM
Articles
- Page Path
- HOME > Ann Occup Environ Med > Volume 37; 2025 > Article
- Special Article Occupational stress (KOSS®19): scale development and validation in the Korean context
-
Hansoo Song1
, Hyoung Ryoul Kim2, Inah Kim3, Jin-Ha Yoon4, Sang-Baek Koh5, Sung-Soo Oh6, Hee-Tae Kang7, Da-Yee Jeung8, Dae-Sung Hyun9, Chunhui Suh10, Sei-Jin Chang5,* -
Annals of Occupational and Environmental Medicine 2025;37:e12.
DOI: https://doi.org/10.35371/aoem.2025.37.e12
Published online: May 7, 2025
1Department of Occupational and Environmental Medicine, Chosun University Hospital, Gwangju, Korea
2Department of Occupational and Environmental Medicine, Seoul St. Mary’s Hospital, Seoul, Korea
3Department of Occupational and Environmental Medicine, Hanyang University College of Medicine, Seoul, Korea
4The Institute for Occupational Health, Yonsei University College of Medicine, Seoul, Korea
5Department of Preventive Medicine, Institute of Occupational and Environmental Medicine, Yonsei University Wonju College of Medicine, Wonju, Korea
6Department of Occupational and Environmental Medicine, Institute of Occupational and Environmental Medicine, Wonju Severance Christian Hospital, Yonsei University Wonju College of Medicine, Wonju, Korea
7Department of Occupational and Environmental Medicine, Wonju Severance Christian Hospital, Yonsei University Wonju College of Medicine, Wonju, Korea
8Department of Dental Hygiene, Hanyang Women’s University, Seoul, Korea
9Department of Non-benefits Management, National Health Insurance Service, Wonju, Korea
10Department of Occupational and Environmental Medicine, Institute of Environmental and Occupational Medicine, Inje University Pusan Paik Hospital, Busan, Korea
- *Corresponding author: Sei-Jin Chang Department of Preventive Medicine, Institute of Occupational and Environmental Medicine, Yonsei University Wonju College of Medicine, 20 Ilsan-ro, Wonju26426, Korea E-mail: chang0343@yonsei.ac.kr
• Received: February 11, 2025 • Revised: April 13, 2025 • Accepted: April 22, 2025
© 2025 Korean Society of Occupational & Environmental Medicine
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (https://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
- 917 Views
- 78 Download
Abstract
-
Background The Korean Occupational Stress Scale (KOSS) was developed in 2004. During this time, industrial structures have evolved, and societal awareness of occupational stress has changed. This study aims to develop and validate a revised version of the Korean Occupational Stress Scale (KOSS®19), tailored for workers, reflecting these changes.
-
Methods: The KOSS®19 was developed based on the 26-item KOSS–short form (SF) through a review by eight experts. A survey was conducted including 359 service industry workers, comprising the KOSS®19, Burnout, and Depression scales. The KOSS®19 subscales were restructured, and their reliability and validity were evaluated.
-
Results: The KOSS®19 composed of eight subscales: hazardous physical environment (2 items), high job demand (3 items), insufficient job control (2 items), low social support (2 items), job insecurity (2 items), organizational injustice (4 items), lack of reward (2 items), and work-life imbalance (2 items). The reliability and validity of the KOSS®19 were found to be satisfactory.
-
Conclusions: The KOSS®19 is a suitable tool for assessing occupational stress, effectively replacing the original KOSS and KOSS-SF.
BACKGROUND
Approximately 970 million individuals globally were living with mental disorders as of 2019, making these disorders the seventh leading cause of disability-adjusted life years. Mental health disparities are significant across regions, sexes, and age groups.1 In South Korea, the 2021 National Mental Health Survey reported a lifetime prevalence of mental disorders at 27.8%, yet the treatment rate for depression and anxiety disorders remained alarmingly low at 9.1%.2
This study focused on workplace mental health, as approximately 15% of working-age adults experience mental disorders at some point, representing a substantial proportion of the population. Mental health issues often hinder workforce participation and lead to exclusion from employment.3 In South Korea, approximately 21% of workers are exposed to high levels of occupational stress, which contributes to 13.6% of major depressive disorders, 4% of suicides, and 6.7% and 6.9% of ischemic heart disease and stroke, respectively.4 Occupational stress adversely affects workers’ health and organization’s productivity.5 The connection between occupational stress and productivity loss aligns with studies demonstrating its association with presenteeism, a critical indicator of pre-emptive productivity decline.6
A clustering analysis of workplace suicide cases reported under Korea's industrial accident insurance identified five categories that link occupational stress to the most severe mental health outcomes: suicide, responsibility burden, role transition, hazard exposure, job insecurity, and workplace violence.7 To address health problems caused by occupational stress, it is essential to use reliable and valid measurement tools. However, occupational stress is a complex and multidimensional concept influenced by factors such as era, nation, culture, age, sex, and occupation. These variations make it challenging to develop standardized tools comparable across contexts.
Internationally, several tools have been adopted to measure occupational stress, including Japan’s Brief Job Stress Questionnaire,8 the United States’ Quality of Work Life Questionnaire,9,10 the United Kingdom’s Work-Related Quality of Life Scale,11 and the Copenhagen Psychosocial Questionnaire, which is widely used across Europe, although it was initially developed in Denmark.12
In South Korea, the Korean Occupational Stress Scale (KOSS), developed by Chang et al. in 2005,13 has been widely used as a standardized tool. According to Crossref, this article has been cited 344 times, reflecting the high level of interest in occupational stress research. However, nearly two decades have passed since the development of this tool. During this time, industrial structures have evolved, and factors such as emotional labor and high-intensity work associated with platform-mediated employment have emerged as significant social challenges.14,15
The concept of lifelong employment as a means of self-realization and job engagement has diminished, and workplace cultures have shifted from competitive, performance-driven environments to those emphasizing fair evaluation, particularly with the emergence of younger generations in the workforce.16,17 Additionally, societal and cultural trends focusing on work-life balance have gained prominence as strategies to address occupational stress.18
This study aimed to revise existing occupational stress measurement tools to reflect these societal changes. Over time, the authors gathered diverse feedback regarding improvements to KOSS. The revision seeks to enhance usability by reducing the number of items and redefining concepts in areas such as workplace culture, organizational injustice, insufficient rewards. The study aims to account for factors that have changed significantly over the years.
METHODS
This study is part of a project titled “Improvement and Utilization of Stress Evaluation Tools for Emotional Labor Workers,” commissioned by the Korea Occupational Safety and Health Agency. The purpose of this research was to develop a revised version of the Korean Occupational Stress Scale (KOSS®19) that could be universally applied to service workers. The initial versions of KOSS-43 and KOSS–short form (SF) were developed through focus group interview and in-depth interviews targeting various occupations in the previous study.13 Starting with the KOSS-SF consisting of 26 items, we organized expert meetings to draft an initial version and subsequently assess its reliability and validity.
To develop a draft of the revised KOSS, the authors refined and finalized the survey items through research team workshops, Delphi surveys, and advisory committee meetings. In the Delphi survey, experts evaluated whether to preserve, modify, or exclude each item. Positive responses regarding preserved or modified items were used to calculate the content validity ratio (CVR). Applying Lawshe’s formula,19 we determined that a CVR of 0.75 or higher indicated a high level of consensus among experts. Through this process, we finalized a revised draft, officially designated as KOSS®19.
To validate KOSS®19, a survey targeting service workers was conducted (n = 370). The survey included demographic information (sex, age, and occupation), the KOSS®19, and measures for depression (Patient Health Questionnaire-9 [PHQ-9]),20 and burnout (the emotional exhaustion subscale of the Maslach Burnout Inventory [MBI]).21 We selected service industry workers engaged in various levels of emotional labor across different sectors and administered face-to-face surveys to collect data for analysis.
We performed an exploratory factor analysis (EFA) to examine the factor structure of the survey items and determine whether the data aligned with the hypothesized factors. The number of factors was determined using eigenvalues of one or higher from the correlation matrix of the observed variables (survey items). Orthogonal rotation via Varimax was applied to calculate the factor loadings and assess whether each survey item aligned with its initially hypothesized factor.
To evaluate model fitness in the factor analysis, we calculated the Kaiser-Meyer-Olkin value, considering values above 0.90 as an excellent fit.22 We also performed Bartlett’s test of sphericity, deeming a p-value greater than 0.05 as an indication of insufficient sample size relative to the number of items.23
Criterion validity was assessed using depression and burnout as outcome variables, with eight subscales as independent variables. Pearson’s correlation analysis was used to calculate correlation coefficients. Convergent and discriminant validity were evaluated using a multitrait-multimethod matrix to derive correlation coefficients.24
To assess internal consistency (reliability), Cronbach’s alpha was calculated, with a value of 0.7 or higher considered indicative of high reliability among the measured items.23 Data were analyzed using IBM SPSS Statistics version 23.0 (IBM Corp., Armonk, NY, USA).
RESULTS
The draft of KOSS®19, as determined through expert consensus, is shown in Table 1. For the “hazardous physical environment”, two items were retained. For “high job demand,” the items “time pressure” and “increased workload” were retained, while “sufficient rest” and “multi-tasking” were excluded. The reasons for exclusion are as follows: “insufficient rest” and “time pressure” were considered redundant, and “multi-tasking” was judged to be either an inherent characteristic of the job or a result of higher job positions.
In the “insufficient job control”, “decision authority” and “workload and schedule control” were retained, while “requires creativity” and “skill discretion” were excluded. The reasons for exclusion are as follows: “creativity” was originally a question about authority in job control but was considered relevant only for certain jobs, with subjective interpretations of creativity varying greatly. “Skill discretion” was likely applicable only to professionals; therefore, it was excluded.
In the “low social support,” “supervisor support,” and “coworker support” were retained, while “empathic support” was excluded. This question was deemed useful only in stressful situations, and similar to “coworker support,” it was considered lower priority. Although the CVR for “supervisor support” was 0.5, which did not meet the selection criteria—it was retained as a core element of support.
The “job insecurity” retained the two items, “job insecurity” and “expecting downsizing,” without modification.
The “organizational injustice” was restructured as the “workplace culture” and “organizational structure” from the original KOSS. “Equitable human resource (HR) system,” “adequate resources available,” and “communication channels” were retained, while “interdepartmental cooperation” was excluded. The reason for exclusion was that, depending on the occupation and size of the company, some workplaces may not have interactions between departments, making the generalization of this item difficult.
In the “lack of rewards,” “socioemotional reward” was retained, while “driven by future prospects” and “opportunities for growth” were excluded. These two items were based on the assumptions of future opportunities and growth and were considered difficult to generalize across occupations. Additionally, a new item was added to reflect “financial reward”—one of the key components of the effort-reward imbalance model.
The three culture-related items in the KOSS reflect South Korea’s unique workplace culture. Since the development of KOSS, the more universally accepted concept of organizational justice has absorbed these elements. Therefore, “uncomfortable team dinner,” “ambiguous job instructions,” “authoritarian workplace culture,” and “sex-based disadvantages” were excluded from KOSS®19.
We reviewed the retention, revision, and deletion of existing items through the Delphi study. The introduction of new items was finalized during the researcher team meeting, after deriving the subscales and items through the literature review. The new items included in this revision are work-life balance, organizational overcommitment, and psychosocial safety climate. The two-items of work-life balance was constructed into new subscale of “work-life imbalance.” Additionally, organizational overcommitment was included as “high job demand,” and psychosocial safety culture was included as "organizational injustice," respectively.
To confirm validity and reliability, a self-administered survey (n = 359) was performed. Regarding the sex distribution of the participants, 114 were male (31.8%) and 245 were female (68.2%). The average age of the respondents was 36.9 years. Regarding occupational status, the largest group consisted of healthcare workers (29.4%), followed by administrative/public service workers (20.9%), call center agents (20.6%), retail and food service workers (10.3%), and police/firefighters (7.0%).
Factorial validity
EFA was conducted on the results of the survey administered using the draft version of KOSS®19. Five factors were identified based on eigenvalues >1. The total variance explained by these five factors is 61.5%. After performing an orthogonal rotation to distinguish the factors, item 19 (safety climate) was grouped with justice, reward, and balance, while the single item for organizational overcommitment was grouped with working conditions and demand (Table 2). The remaining items were grouped according to the initial hypotheses. The authors discussed these results and finalized the KOSS®19 as described below.
Factor 1 included eight items (11–19) that were grouped together under the same component. However, the originally proposed domains were maintained, and the items were re-categorized as “organizational injustice” (items 11, 12, 13, and 19), “lack of reward” (items 14 and 15), and “work-life imbalance” (items 17 and 18). Factor 2 included items 1, 2, 3, and 4, as well as the newly proposed item 16 (organizational overcommitment). The initial subdomains were preserved, with “hazardous physical environment” (items 1 and 2) retained, and item 16 (organizational overcommitment) included in the “high job demand” domain (items 3, 4, and 16). Factor 3 included “low social support” (items 7 and 8), and this grouping was maintained. Factor 4, grouped under “job insecurity” (items 9 and 10), was retained. Factor 5 was grouped under “insufficient control” (items 5 and 6), and this categorization remained unchanged. Ultimately, the 19 items in the KOSS®19 were re-categorized into eight subscales.
Criterion validity
A correlation analysis was conducted between the subscales of the final KOSS®19 and burnout and depression. The results showed significant positive correlations between burnout, depression, and all subscales, except for “low social support.” Among the subscales, “low social support” had the weakest correlations, with burnout showing a statistically significant correlation and depression not showing a significant correlation (Table 3).
Discriminant validity
To evaluate the appropriateness of convergence and discriminant validity between the pre-conceptualized factors and the subscales in the final KOSS®19, a multitrait-multimethod matrix analysis was conducted (Table 4). Convergent validity was established when the correlation between each item and its corresponding factor was at least 0.40, whereas discriminant validity was established when the correlation between each item and its corresponding factor was greater than the correlation between the item and the subscales to which it did not belong. The success rate of item discriminant validity was calculated as the percentage of successful tests out of the total number. The results showed that convergent validity was strong for all eight subdomains, with correlation coefficients (r) above 0.7 and a scaling success rate of 100%. Furthermore, the success rate for discriminant validity was 100% for all the subscales (Table 5).
The reliability analysis for the final KOSS®19 indicated satisfactory reliability, with Cronbach’s alpha values ranging from 0.592 to 0.779 (Table 6).
DISCUSSION
In this study, a KOSS®19 questionnaire consisting of 19 items across eight subscales, was developed. The subscales include: “hazardous physical environment” (2 items), “high job demand” (3 items), “insufficient control” (2 items), “low social support” (2 items), “job insecurity” (2 items), “organizational injustice” (4 items), “lack of reward” (2 items), and “work-life imbalance” (2 items). The results of the reliability and validity assessments of the service industry workers were satisfactory.
The EFA revealed five factors. Among the eight subscales, “insufficient control,” “low social support,” and “job insecurity” were clearly distinguished, while “organizational injustice,” “lack of reward,” and “work-life imbalance” were grouped into one subscale and “risky physical environment” and “high job demand” were grouped together. Possible explanations for this issue include items failing to differentiate between different concepts, when there are high correlations between concepts, when the sample size is small, or when the items lack commonality based on the latent variables that they aim to measure. According to the theoretically hypothesis-driven original KOSS study model, we divided it into six existing factors (hazardous physical environment, high job demand, insufficient control, low social support, job insecurity, lack of reward), one modified factor (organizational injustice), and one additional factor (work-life imbalance). Additionally, we performed confirmatory factor analysis (not shown in Table), and found that they are reclassified into eight subscales. In this study, since the factor loadings and discriminant validity were relatively good, the close correlation between factors may have caused this issue. The sample size in this study was insufficient to analyze by occupation, suggesting that a larger-scale study that includes various job types is necessary. The study was conducted only on service sector workers and on a specific dataset of 370 participants. Further research is needed in other datasets targeting various occupations to ensure the generalizability.
This study did not aim to choose the best theoretical model among many but rather focused on identifying how individuals exposed to occupational stressors can be categorized while maintaining existing theoretical models. This approach can be an effective way to measure actual occupational stress.25,26 From this perspective, the following observations were made for each of the subscales:
“Hazardous physical environment” reflects an overall subjective perception of the working environment. “Dangerous work” refers to the perceived risk of accidents, and “physical burden” addresses ergonomic risk factors in a single item. Recently, hazardous work environmental factors such as accident risks, extreme temperatures, smoke, and ergonomic risks in sectors such as delivery services and logistics have become more prominent, increasing the importance of this item.27-29
“High job demand” is a classical and core factor of occupational stress. Elements of job demand, such as long working hours and overwork, are used as evidence for cardiovascular disease compensation. However, the effects of “high job demand” are moderated by control, support, and resources; therefore, it should be considered alongside other factors.30 In this tool, the concepts reflected in job demand include time pressure, increased workload, and overcommitment. Time pressure is a key component of other occupational stress assessment tools. An increased workload reflects fluctuations in work volume caused by unexpected events or spontaneous organizational demands, such as those witnessed during the coronavirus disease 2019 pandemic.31,32 Overcommitment, which is either required or encouraged by the organization, is related to cultural traits in Korean companies, such as work centrality, collectivism, future orientation, and strong work ethics.33
Reduced control increases tension when combined with increased job demand. In contrast, an increase in control reportedly improves employee’s mental health.34,35 Task authority and skill discretion are considered elements of control. Task authority refers to personal control over work methods and decision-making processes. Skill discretion refers to how individuals can utilize and control their skills or abilities. In KOSS®19, the item regarding skill discretion was excluded. Control over skill has largely shifted to capital, with labor dependent on platforms based on creativity and technical leadership.36 Questions based on skill discretion are often replaced with issues of “status.” Thus, the final tool reflects control through items regarding decision-making authority and control over workload and schedules, which can be generalized to all workers and used to improve occupational stress.
“Low social support” refers to support from supervisors and coworkers, a core concept in Karasek and Theorell’s demand-control model.37 The “buffer hypotheses” have shown consistent results in studies.26 Supervisor support affects attitudes toward work, whereas coworker support reflects practical and immediate assistance, explaining the two aspects of support well.38
“Job insecurity” refers to the degree of job or career stability, including concerns about job insecurity and employment instability.39 Downsizing during economic recessions and restructuring due to pandemics have been reported to negatively affect workers’ mental health.40 The items proposed in KOSS®19 reflect individuals’ concerns about unemployment or job deterioration prospects. According to existing studies, perceived job insecurity independently impacts health.41 The instability resulting from role restructuring due to workforce reduction or high turnover rates and the psychological response of survivors after layoffs narrates a similar issue.42
In KOSS®19, “organizational injustice” is composed of four items and is proposed in three dimensions: distributive, procedural, and interactional justice.43 Multiple discussions occurred regarding the redundancy between items in different domains. For example, distributive justice is addressed in an effort-reward imbalance model. “Organizational injustice” is related to supervisor and coworker support and organizational incivility.44 However, procedural and interactional justice, as part of organizational justice, negatively affects mental health independently from other factors.45 Therefore, to maintain the discriminant validity of the items, organizational justice in KOSS®19 was constructed only with the procedural and interactional justice dimensions, excluding overlapping items related to supervisor and coworker support. In this survey, procedural rationality in organizational decision-making was reflected through an equitable HR system and communication channels, whereas interactional justice was reflected through adequate resources and a safety climate. Organizational injustice is not a subjective perception of individuals and is not directly related to supervisor support. This evaluation item reflects an overall organizational assessment and addresses a dimension distinct from other factors.
“Lack of reward” plays a central role in the effort-reward imbalance model. Rewards are divided into monetary compensation and non-monetary rewards, such as respect, opportunities for development, and job security.46-48 This survey evaluated both appropriate financial rewards and socioemotional rewards.
“Work-life imbalance” has become widely accepted in recent years. This concept reveals differences in perspectives between older generations and newer generations regarding the value of work.49 Previously, work-family conflict focused on how excessive workloads affected the family life of married individuals.50 As a result, work-life imbalance was used to reflect job demands. However, since interference with personal life is emphasized as a unique stressor when work-life imbalance is appropriately managed, work enhances life. Thus, the two items in the balance domain demonstrated appropriate internal consistency and discriminant validity.
When using KOSS®19, it is not recommended to use the total score by summing all items or calculating the overall mean score. The primary purpose of assessing occupational stress is to identify vulnerability factors. The use of summed or averaged scores may hinder the identification of these critical factors. Furthermore, the total score does not necessarily represent the overall quantity of stress.
This study had several limitations. First, the participants used to validate the reliability and validity of the tool were from various service industries. Therefore, the generalizability of these findings to primary industries or the manufacturing sectors cannot be confirmed. It is required to perform the further validation study targeting various occupations. Second, since the validity assessment was conducted through a cross-sectional study, the generalizability of its findings is limited. While the latent period between exposure to occupational stress and its outcomes is not long, which makes it acceptable, further evaluation through a prospective study is necessary. Although the internal consistency was good, certain items had Cronbach’s alpha values below 0.6, which were not satisfactory. Additional reviews of individual survey items, item responses, difficulty of answers, and the appropriateness of meaning delivery may be required. Despite these limitations, the strength of this tool lies in its ability to reflect the characteristics of Korean’s organizational culture and the evolving perceptions of occupational stress in response to social changes. Additionally, this study provides reference values based on sex, making it practical for application in workplace stress management (Supplementary Tables 1 and 2). A Korean version of KOSS®19 was provided (Supplementary Data 1).
CONCLUSIONS
KOSS®19, an occupational stress assessment tool composed of 19 items across 8 subscales, demonstrated relatively good reliability and validity. This study focused on verifying existing theoretical models and considering the actual exposure to occupational stress, reviewing the redundancy between concepts, and constructing a questionnaire with mutually independent factors. Improving the items from the original KOSS, or short version, excluding unnecessary items, and adding new ones better reflects the occupational stress experienced by current service industry workers. Therefore, this tool can replace the existing full version or short-form of KOSS.
Further research is necessary to determine whether this tool can be appropriately applied across various industries. Assessing improvements before and after occupational stress interventions will be an important task for future research.
Abbreviations
CVR
content validity ratio
EFA
exploratory factor analysis
HR
human resource
KOSS
Korean Occupational Stress Scale
MBI
Maslach Burnout Inventory
SF
short form
PHQ-9
Patient Health Questionnaire-9
-
Funding
This study was supported by Occupational Safety and Health Research Institute (2018-OSHRI-790).
-
Competing interests
Hansoo Song, Hyoung Ryoul Kim, Inah Kim, Jin-Ha Yoon, Sang-Baek Koh, Sung-Soo Oh, and Chunhui Suh contributing editors of the Annals of Occupational and Environmental Medicine, were not involved in the editorial evaluation or decision to publish this article. All remaining authors have declared no conflicts of interest.
-
Author contributions
Conceptualization: Chang SJ. Data curation: Hyun DS, Jeung DY. Formal analysis: Kim HR, Kim I, Yoon JH. Investigation: Kim I, Koh SB. Methodology: OH SS, Kang HT. Validation: Song H, Suh C. Writing - original draft: Song H, Chang SJ. Writing - review & editing: Song H.
NOTES
SUPPLEMENTARY MATERIAL
Supplementary Table 1.
ROC analysis and cut-off value of subscales based on depression by sex.
Supplementary Table 2.
Proposed reference value of subscales by sex.
Table 1.
Delphi survey results: selection and refinement of revised items in KOSS R-26
No. of panelists = 8, minimum value of construct validity ratio = 0.75.
KOSS: Korean Occupational Stress Scale; P: preserved; M: modified; E: excluded; CVR: construct validity ratio; EN: hazardous physical environment; DE: high job demand; CO: insufficient control; SU: low social support; SE: job insecurity; JU: organizational injustice; RE: lack of reward; CU: negative workplace culture; OC: overcommitment; BA: work-life imbalance; CL: unsafety climate.
Table 2.
Rotated component matrix
Table 3.
Correlations between KOSS®19 subscales and burnout/depression
Table 4.
Multitrait-multimethod matrix for assessing item convergence and discrimination
| Subscale | No. of items |
Correlation coefficient interval |
Scaling success, rate (%)a |
||
|---|---|---|---|---|---|
| Convergent validityb | Discriminant validityc | Convergent validity | Discriminant validity | ||
| EN | 2 | 0.831–0.857 | 0.036–0.508 | 2/2 (100) | 16/16 (100) |
| DE | 3 | 0.743–0.865 | 0.068–0.586 | 3/3 (100) | 24/24 (100) |
| CO | 2 | 0.813–0.851 | 0.022–0.350 | 2/2 (100) | 16/16 (100) |
| SU | 2 | 0.810–0.891 | 0.009–0.366 | 2/2 (100) | 16/16 (100) |
| SE | 2 | 0.884–0.887 | 0.053–0.353 | 2/2 (100) | 16/16 (100) |
| JU | 4 | 0.751–0.805 | 0.062–0.558 | 4/4 (100) | 32/32 (100) |
| RE | 2 | 0.809–0.873 | 0.058–0.662 | 2/2 (100) | 16/16 (100) |
| BA | 2 | 0.881–0.882 | 0.104–0.500 | 2/2 (100) | 16/16 (100) |
KOSS: Korean Occupational Stress Scale; EN: hazardous physical environment; DE: high job demand; CO: insufficient control; SU: low social support; SE: job insecurity; JU: organizational injustice; RE: lack of reward; BA: work-life imbalance.
aScaling success: No. of convergent correlations significantly higher than discriminant correlation/No. of correlations;
bCorrelations with own scale;
cCorrelations with other scales (range of correlations).
Table 5.
Results of item-scaling tests (n = 359)
Table 6.
Final KOSS®19 questionnaire and reliability estimates (n = 359)
- 1. GBD 2019 Mental Disorders Collaborators. Global, regional, and national burden of 12 mental disorders in 204 countries and territories, 1990-2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet Psychiatry 2022;9(2):137–50.ArticlePubMedPMC
- 2. Rim SJ, Hahm BJ, Seong SJ, Park JE, Chang SM, Kim BS, et al. Prevalence of mental disorders and associated factors in Korean adults: National Mental Health Survey of Korea 2021. Psychiatry Investig 2023;20(3):262–72.ArticlePubMedPMCPDF
- 3. Rugulies R, Aust B, Greiner BA, Arensman E, Kawakami N, LaMontagne AD, et al. Work-related causes of mental health conditions and interventions for their improvement in workplaces. Lancet 2023;402(10410):1368–81.ArticlePubMed
- 4. Lee K, Kim I. Job stress-attributable burden of disease in Korea. J Korean Med Sci 2018;33(25):e187.ArticlePubMedPMCPDF
- 5. Chung J, Kim JH, Lee JY, Kang HS, Lee DW, Hong YC, et al. The association between occupational stress level and health-related productivity loss among Korean employees. Epidemiol Health 2023;45:e2023009.ArticlePubMedPDF
- 6. Kim J, Kim YK, Leem SH, Won JU. Association between job-related stress and experience of presenteeism among Korean workers stratified on the presence of depression. Ann Occup Environ Med 2019;31:e26.ArticlePubMedPMCPDF
- 7. Jang J, Kim I, Kim Y, Song J. Comparison of work-related stress in cluster of workers' suicides in Korea: analysis of industrial accident compensation insurance, 2010-2017. Int J Environ Res Public Health 2022;19(5):3013.ArticlePubMedPMC
- 8. Watanabe K, Imamura K, Eguchi H, Hidaka Y, Komase Y, Sakuraya A, et al. Usage of the brief job stress questionnaire: a systematic review of a comprehensive job stress questionnaire in Japan from 2003 to 2021. Int J Environ Res Public Health 2023;20(3):1814.ArticlePubMedPMC
- 9. National Institute for occupational safety and Health. Quality of worklife questionnaire. https://www.cdc.gov/niosh/stress/worklife-question/index.html. Updated 2024. Accessed January 15, 2025
- 10. Choi B, Seo Y. Developing a short standard questionnaire for assessing work organization hazards: the Healthy Work Survey (HWS). Ann Occup Environ Med 2023;35:e7.ArticlePubMedPMCPDF
- 11. Van Laar D, Edwards JA, Easton S. The work-related quality of life scale for healthcare workers. J Adv Nurs 2007;60(3):325–33.ArticlePubMed
- 12. Kristensen TS, Hannerz H, Hogh A, Borg V. The Copenhagen Psychosocial Questionnaire: a tool for the assessment and improvement of the psychosocial work environment. Scand J Work Environ Health 2005;31(6):438–49.ArticlePubMed
- 13. Chang SJ, Koh SB, Kang D, Kim SA, Kang MG, Lee CG, et al. Developing an occupational stress scale for Korean employees. Korean J Occup Environ Med 2005;17(4):297–317.Article
- 14. Kim MS, Oh J, Sim J, Yun BY, Yoon JH. Association between exposure to violence, job stress and depressive symptoms among gig economy workers in Korea. Ann Occup Environ Med 2023;35:e43.ArticlePubMedPMCPDF
- 15. Lee G. Korean emotional laborers' job stressors and relievers: focus on work conditions and emotional labor properties. Saf Health Work 2015;6(4):338–44.ArticlePubMedPMC
- 16. Goh T, Lim H, Wang J, Choi D, Lee C. Generational culture and organizational culture: focusing on work satisfaction and cultural differences between generations in the Korean businesses. Korean J Cult Sociol 2021;29(2):39–96.Article
- 17. Cho HK. An exploratory study on Korean workers’ cognition of career success: basic research for international comparison. J Skills Qualif 2018;7(2):197–220.Article
- 18. Yang JW, Suh C, Lee CK, Son BC. The work-life balance and psychosocial well-being of South Korean workers. Ann Occup Environ Med 2018;30:38.ArticlePubMedPMCPDF
- 19. Lawshe CH. A quantitative approach to content validity. Pers Psychol 1975;28(4):563–75.Article
- 20. An JY, Seo ER, Lim KH, Shin JH, Kim JB. Standardization of the Korean version of screening tool for depression (patient health questionnaire-9, PHQ-9). J Korean Soc Biol Ther Psychiatry 2013;19(1):47–56.
- 21. Shin KH. The Maslach burnout inventory-general survey (MBI-GS): an application in South Korea. Korean J Ind Organ Psychol 2003;16(3):1–17.
- 22. Boateng GO, Neilands TB, Frongillo EA, Melgar-Quinonez HR, Young SL. Best practices for developing and validating scales for health, social, and behavioral research: a primer. Front Public Health 2018;6:149.ArticlePubMedPMC
- 23. Kaiser HF. An index of factorial simplicity. Psychometrika 1974;39(1):31–6.ArticlePDF
- 24. Schmitt N, Stults DM. Methodology review: analysis of multitrait-multimethod matrices. Appl Psychol Meas 1986;10(1):1–22.ArticlePDF
- 25. Portoghese I, Galletta M, Leiter MP, Finco G, d'Aloja E, Campagna M. Job demand-control-support latent profiles and their relationships with interpersonal stressors, job burnout, and intrinsic work motivation. Int J Environ Res Public Health 2020;17(24):9430.ArticlePubMedPMC
- 26. Gameiro M, Chambel MJ, Carvalho VS. A person-centered approach to the job demands-control model: a multifunctioning test of addictive and buffer hypotheses to explain burnout. Int J Environ Res Public Health 2020;17(23):8871.ArticlePubMedPMC
- 27. Lu Y, Zhang Z, Yan H, Rui B, Liu J. Effects of occupational hazards on job stress and mental health of factory workers and miners: a propensity score analysis. Biomed Res Int 2020;2020:1754897.ArticlePubMedPMCPDF
- 28. Yoo H, Yang M, Song JH, Yoon JH, Lee W, Jang J, et al. Investigation of working conditions and health status in platform workers in the Republic of Korea. Saf Health Work 2024;15(1):17–23.ArticlePubMedPMC
- 29. Zheng Q, Zhan J, Feng X. Working safety and workloads of Chinese delivery riders: the role of work pressure. Int J Occup Saf Ergon 2023;29(2):869–82.ArticlePubMed
- 30. Nahrgang JD, Morgeson FP, Hofmann DA. Safety at work: a meta-analytic investigation of the link between job demands, job resources, burnout, engagement, and safety outcomes. J Appl Psychol 2011;96(1):71–94.ArticlePubMed
- 31. Ryoo SW, Min JY, Son SY, Choi BY, Choi J, Min KB. Telecommuting during the COVID-19: the role of job demand and control on health outcomes. Ann Occup Environ Med 2024;36:e14.ArticlePubMedPMCPDF
- 32. Scheel TE, Bendixen L, Prochazka J, Acquadro Maran D. Working during the COVID-19 pandemic: demands, resources, and mental wellbeing. Front Psychol 2022;13:1037866.ArticlePubMed
- 33. Kim S, McLean GN, Park S. The cultural context of long working hours: workplace experiences in Korea. New Horiz Adult Educ Hum Resour Dev 2018;30(2):36–51.ArticlePDF
- 34. Egan M, Bambra C, Thomas S, Petticrew M, Whitehead M, Thomson H. The psychosocial and health effects of workplace reorganisation. 1. A systematic review of organisational-level interventions that aim to increase employee control. J Epidemiol Community Health 2007;61(11):945–54.ArticlePubMedPMC
- 35. Bambra C, Egan M, Thomas S, Petticrew M, Whitehead M. The psychosocial and health effects of workplace reorganisation. 2. A systematic review of task restructuring interventions. J Epidemiol Community Health 2007;61(12):1028–37.ArticlePubMedPMC
- 36. Glavin P, Schieman S. Dependency and hardship in the gig economy: the mental health consequences of platform work. Socius 2022;8:23780231221082414.ArticlePDF
- 37. Karasek R, Theorell T. Healthy Work: Stress, Productivity, and the Reconstruction of Working Life. New York, NY: Basic Books; 1990.
- 38. Fila MJ, Purl J, Griffeth RW. Job demands, control and support: meta-analyzing moderator effects of gender, nationality, and occupation. Hum Resour Manag Rev 2017;27(1):39–60.Article
- 39. Sora B, Hoge T, Caballer A, Peiro JM. The construct of job insecurity at multiple levels: implications for its conceptualization and theory development. Int J Environ Res Public Health 2023;20(4):3052.ArticlePubMedPMC
- 40. Ervasti J, Aalto V, Pentti J, Oksanen T, Kivimaki M, Vahtera J. Association of changes in work due to COVID-19 pandemic with psychosocial work environment and employee health: a cohort study of 24 299 Finnish public sector employees. Occup Environ Med 2022;79(4):233–41.ArticlePubMed
- 41. Virtanen M, Nyberg ST, Batty GD, Jokela M, Heikkila K, Fransson EI, et al. Perceived job insecurity as a risk factor for incident coronary heart disease: systematic review and meta-analysis. BMJ 2013;347:f4746.ArticlePubMedPMC
- 42. Modrek S, Cullen MR. Job insecurity during recessions: effects on survivors' work stress. BMC Public Health 2013;13:929.ArticlePubMedPMCPDF
- 43. Ndjaboue R, Brisson C, Vezina M. Organisational justice and mental health: a systematic review of prospective studies. Occup Environ Med 2012;69(10):694–700.ArticlePubMed
- 44. Namin BH, Ogaard T, Roislien J. Workplace incivility and turnover intention in organizations: a meta-analytic review. Int J Environ Res Public Health 2021;19(1):25.ArticlePubMedPMC
- 45. Park H, Lee KS, Park YJ, Lee DJ, Lee HK. Reliability and validity of the Korean version of organizational justice questionnaire. Ann Occup Environ Med 2018;30:26.ArticlePubMedPMCPDF
- 46. Jachens L, Houdmont J. Effort-reward imbalance and job strain: a composite indicator approach. Int J Environ Res Public Health 2019;16(21):4169.ArticlePubMedPMC
- 47. van Vegchel N, de Jonge J, Bosma H, Schaufeli W. Reviewing the effort-reward imbalance model: drawing up the balance of 45 empirical studies. Soc Sci Med 2005;60(5):1117–31.ArticlePubMed
- 48. Siegrist J, Li J. Associations of extrinsic and intrinsic components of work stress with health: a systematic review of evidence on the effort-reward imbalance model. Int J Environ Res Public Health 2016;13(4):432.ArticlePubMedPMC
- 49. Baek SU, Lee YM, Yoon JH, Won JU. Long working hours, work-life imbalance, and poor mental health: a cross-sectional mediation analysis based on the sixth Korean working conditions survey, 2020-2021. J Epidemiol 2024;34(11):535–42.ArticlePubMedPMC
- 50. Gragnano A, Simbula S, Miglioretti M. Work-life balance: weighing the importance of work-family and work-health balance. Int J Environ Res Public Health 2020;17(3):907.ArticlePubMedPMC
REFERENCES
Figure & Data
REFERENCES
Citations
Citations to this article as recorded by 
Cite
XML DownloadOccupational stress (KOSS®19): scale development and validation in the Korean context
Occupational stress (KOSS®19): scale development and validation in the Korean context
| Subscale | Question | Delphi |
Conclusions (#Order of selected questions) | |||
|---|---|---|---|---|---|---|
| P | M | E | CVR | |||
| EN | Dangerous work | 6 | 2 | 0 | 1 | #1 |
| Physical burden | 6 | 1 | 1 | 0.75 | #2 | |
| DE | Time pressure | 8 | 0 | 0 | 1 | #3 |
| Increased workload | 6 | 1 | 1 | 0.75 | #4 | |
| Sufficient rest | 1 | 0 | 7 | –0.75 | Exclude | |
| Multi-tasking | 2 | 0 | 6 | –0.5 | Exclude | |
| CO | Requires creativity | 0 | 0 | 8 | –1 | Exclude |
| Skill Discretion | 2 | 1 | 5 | –0.25 | Exclude | |
| Decision authority | 7 | 1 | 0 | 1 | #5 | |
| Workload and schedule control | 7 | 0 | 1 | 0.75 | #6 | |
| SU | Supervisor support | 0 | 6 | 2 | 0.5 | #7 |
| Coworker support | 0 | 7 | 1 | 0.75 | #8 | |
| Empathetic support | 3 | 0 | 5 | –0.25 | Exclude | |
| SE | Job insecurity | 8 | 0 | 0 | 1 | #9 |
| Expected downsizing | 7 | 0 | 1 | 0.75 | #10 | |
| JU | Equitable human resource system | 7 | 1 | 0 | 1 | #11 |
| Adequate resources available | 8 | 0 | 0 | 1 | #12 | |
| Interdepartmental cooperation | 0 | 0 | 8 | –1 | Exclude | |
| Communication channels | 8 | 0 | 0 | 1 | #13 | |
| RE | Socioemotional reward | 7 | 1 | 0 | 1 | #15 |
| Driven by future prospects | 0 | 0 | 8 | –1 | Exclude | |
| Opportunities for growth | 2 | 1 | 5 | –0.25 | Exclude | |
| CU | Uncomfortable team dinner | 0 | 0 | 8 | –1 | Exclude |
| Ambiguous job instructions | 0 | 0 | 8 | –1 | Exclude | |
| Authoritarian workplace culture | 0 | 1 | 7 | –1 | Exclude | |
| RE | Financial rewards | #14 | ||||
| OC | Organizational overcommitment | #16 | ||||
| CU | Gender-based disadvantages | 0 | 0 | 8 | –1 | Exclude |
| BA | Work-life imbalance | #17 | ||||
| Life-enhancing work | #18 | |||||
| CL | Safety climate | #19 | ||||
| Order | Items | Component |
||||
|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | ||
| 18 | Life-enhancing work | 0.767 | 0.103 | –0.068 | 0.049 | 0.017 |
| 19 | Safety climate | 0.706 | 0.192 | 0.191 | 0.055 | 0.110 |
| 14 | Appropriate financial rewards | 0.703 | 0.209 | –0.032 | –0.099 | 0.081 |
| 15 | Socioemotional reward | 0.634 | 0.069 | 0.360 | –0.001 | 0.242 |
| 12 | Adequate resource available | 0.626 | 0.316 | 0.258 | 0.106 | 0.046 |
| 17 | Work-life imbalance | 0.612 | 0.333 | –0.043 | 0.218 | 0.065 |
| 11 | Equitable human resource system | 0.543 | 0.255 | 0.371 | –0.120 | 0.120 |
| 13 | Communication | 0.464 | 0.108 | 0.454 | 0.147 | 0.392 |
| 3 | Time pressure | 0.187 | 0.841 | 0.034 | 0.083 | 0.063 |
| 4 | Increased overload | 0.249 | 0.778 | 0.033 | 0.063 | –0.042 |
| 1 | Dangerous work | 0.216 | 0.642 | –0.038 | 0.021 | –0.042 |
| 2 | Physical burden | 0.180 | 0.633 | –0.036 | 0.221 | 0.255 |
| 16 | Organizational overcommitment | 0.141 | 0.532 | 0.176 | 0.057 | 0.162 |
| 7 | Supervisor support | 0.071 | 0.138 | 0.811 | –0.100 | 0.147 |
| 8 | Coworker support | 0.110 | –0.131 | 0.758 | 0.163 | –0.030 |
| 9 | Job insecurity | 0.109 | 0.091 | 0.025 | 0.876 | 0.037 |
| 10 | Expecting downsizing | –0.051 | 0.290 | 0.049 | 0.809 | 0.035 |
| 5 | Decision authority | 0.090 | –0.049 | 0.102 | –0.031 | 0.833 |
| 6 | Workload and schedule control | 0.169 | 0.230 | 0.051 | 0.097 | 0.756 |
| Subscale | Burnout |
Depression |
||
|---|---|---|---|---|
| Correlation coefficient | p-value | Correlation coefficient | p-value | |
| EN | 0.434 | <0.001 | 0.345 | <0.001 |
| DE | 0.658 | <0.001 | 0.437 | <0.001 |
| CO | 0.278 | <0.001 | 0.150 | 0.004 |
| SU | 0.077 | <0.001 | 0.014 | 0.793 |
| SE | 0.264 | <0.001 | 0.236 | <0.001 |
| JU | 0.432 | <0.001 | 0.358 | <0.001 |
| RE | 0.341 | <0.001 | 0.352 | <0.001 |
| BA | 0.459 | <0.001 | 0.462 | <0.001 |
| Subscale | No. of items | Correlation coefficient interval |
Scaling success, rate (%) |
||
|---|---|---|---|---|---|
| Convergent validity |
Discriminant validity |
Convergent validity | Discriminant validity | ||
| EN | 2 | 0.831–0.857 | 0.036–0.508 | 2/2 (100) | 16/16 (100) |
| DE | 3 | 0.743–0.865 | 0.068–0.586 | 3/3 (100) | 24/24 (100) |
| CO | 2 | 0.813–0.851 | 0.022–0.350 | 2/2 (100) | 16/16 (100) |
| SU | 2 | 0.810–0.891 | 0.009–0.366 | 2/2 (100) | 16/16 (100) |
| SE | 2 | 0.884–0.887 | 0.053–0.353 | 2/2 (100) | 16/16 (100) |
| JU | 4 | 0.751–0.805 | 0.062–0.558 | 4/4 (100) | 32/32 (100) |
| RE | 2 | 0.809–0.873 | 0.058–0.662 | 2/2 (100) | 16/16 (100) |
| BA | 2 | 0.881–0.882 | 0.104–0.500 | 2/2 (100) | 16/16 (100) |
| Subscale | Item | Item-scale correlation |
|||||||
|---|---|---|---|---|---|---|---|---|---|
| EN | DE | CO | SU | SE | JU | RE | BA | ||
| EN | 1 | 0.847 | 0.453 | 0.123 | 0.036 | 0.184 | 0.315 | 0.284 | 0.292 |
| 2 | 0.831 | 0.508 | 0.253 | 0.058 | 0.330 | 0.330 | 0.282 | 0.368 | |
| DE | 3 | 0.586 | 0.865 | 0.203 | 0.068 | 0.274 | -0.401 | 0.278 | 0.428 |
| 4 | 0.507 | 0.855 | 0.157 | 0.075 | 0.251 | 0.386 | 0.319 | 0.424 | |
| 5 | 0.297 | 0.742 | 0.210 | 0.104 | 0.298 | 0.365 | 0.297 | 0.244 | |
| CO | 6 | 0.082 | 0.067 | 0.813 | 0.206 | 0.022 | -0.234 | 0.206 | 0.108 |
| 7 | 0.281 | 0.313 | 0.851 | 0.157 | 0.170 | 0.350 | 0.275 | 0.259 | |
| SU | 8 | 0.097 | 0.154 | 0.225 | 0.891 | 0.020 | 0.366 | 0.248 | 0.148 |
| 9 | -0.017 | 0.009 | 0.127 | 0.810 | 0.104 | 0.265 | 0.149 | 0.137 | |
| SE | 10 | 0.256 | 0.237 | 0.096 | 0.056 | 0.887 | 0.210 | 0.104 | 0.194 |
| 11 | 0.279 | 0.353 | 0.103 | 0.063 | 0.884 | 0.144 | 0.053 | 0.152 | |
| JU | 12 | 0.246 | 0.361 | 0.228 | 0.299 | 0.062 | 0.781 | 0.543 | 0.337 |
| 13 | 0.404 | 0.405 | 0.205 | 0.258 | 0.218 | 0.805 | 0.519 | 0.492 | |
| 14 | 0.238 | 0.299 | 0.402 | 0.358 | 0.192 | 0.751 | 0.501 | 0.326 | |
| 15 | 0.304 | 0.393 | 0.270 | 0.263 | 0.146 | 0.774 | 0.558 | 0.526 | |
| RE | 16 | 0.304 | 0.339 | 0.198 | 0.089 | 0.058 | 0.532 | 0.873 | 0.440 |
| 17 | 0.255 | 0.271 | 0.303 | 0.337 | 0.097 | 0.622 | 0.809 | 0.418 | |
| BA | 18 | 0.399 | 0.466 | 0.226 | 0.167 | 0.238 | 0.472 | 0.401 | 0.881 |
| 19 | 0.295 | 0.327 | 0.164 | 0.133 | 0.104 | 0.468 | 0.500 | 0.882 | |
| Initial no. | Final no. | Subscale | Question | Cronbach’s alpha |
|---|---|---|---|---|
| 1 | 1 | EN | Dangerous work | 0.568 |
| 2 | 2 | Physical burden | ||
| 3 | 3 | DE | Time pressure | 0.757 |
| 4 | 4 | Increased overload | ||
| 16 | 5 | Organizational overcommitment | ||
| 5 | 6 | CO | Decision authority | 0.559 |
| 6 | 7 | Workload and schedule control | ||
| 7 | 8 | SU | Supervisor support | 0.609 |
| 8 | 9 | Coworker support | ||
| 9 | 10 | SE | Job insecurity | 0.727 |
| 10 | 11 | Expecting downsizing | ||
| 11 | 12 | JU | Equitable human resource system | 0.779 |
| 12 | 13 | Adequate resource available | ||
| 13 | 14 | Communication channels | ||
| 19 | 15 | Safety climate | ||
| 14 | 16 | RE | Appropriate financial rewards | 0.592 |
| 15 | 17 | Socioemotional reward | ||
| 17 | 18 | BA | Work-life imbalance | 0.712 |
| 18 | 19 | Life-enhancing work |
Table 1. Delphi survey results: selection and refinement of revised items in KOSS R-26
No. of panelists = 8, minimum value of construct validity ratio = 0.75. KOSS: Korean Occupational Stress Scale; P: preserved; M: modified; E: excluded; CVR: construct validity ratio; EN: hazardous physical environment; DE: high job demand; CO: insufficient control; SU: low social support; SE: job insecurity; JU: organizational injustice; RE: lack of reward; CU: negative workplace culture; OC: overcommitment; BA: work-life imbalance; CL: unsafety climate.
Table 2. Rotated component matrix
The bold font is the factor loading value of each item attributed to the subscales.
Table 3. Correlations between KOSS®19 subscales and burnout/depression
KOSS: Korean Occupational Stress Scale; EN: hazardous physical environment; DE: high job demand; CO: insufficient control; SU: low social support; SE: job insecurity; JU: organizational injustice; RE: lack of reward; BA: work-life imbalance.
Table 4. Multitrait-multimethod matrix for assessing item convergence and discrimination
KOSS: Korean Occupational Stress Scale; EN: hazardous physical environment; DE: high job demand; CO: insufficient control; SU: low social support; SE: job insecurity; JU: organizational injustice; RE: lack of reward; BA: work-life imbalance. Scaling success: No. of convergent correlations significantly higher than discriminant correlation/No. of correlations; Correlations with own scale; Correlations with other scales (range of correlations).
Table 5. Results of item-scaling tests (n = 359)
The bold font is the factor loading value of each item attributed to the subscales and each item.
EN: hazardous physical environment; DE: high job demand; CO: insufficient control; SU: low social support; SE: job insecurity; JU: organizational injustice; RE: lack of reward; BA: work-life imbalance.
Table 6. Final KOSS®19 questionnaire and reliability estimates (n = 359)
KOSS: Korean Occupational Stress Scale; EN: hazardous physical environment; DE: high job demand; CO: insufficient control; SU: low social support; SE: job insecurity; JU: organizational injustice; RE: lack of reward; BA: work-life imbalance.
