Introduction
The assessment of growth and development is essential for growing children. Particularly, infants and children are the most important periods for development, and developmental evaluation during this period may reduce the chances of future disorders and prevent secondary sequelae. The most common clinical symptom of neurodevelopmental disorders is the lack of developmental skills consistent with their age. Therefore, if early screening of infants and children who may have future developmental problems is possible through assessment, then a more accurate assessment should be conducted to plan for appropriate treatments and rehabilitation.
National Health Screening Program for Infants and Children (NHSPIC) has been rolled out nationwide since November 2007 to keep track of the growth and development of infants and children and to provide proper education programs to caregivers, with the introduction of a health check-up program suitable for the age of infants and children. NHSPIC is conducted for the first to seventh rounds as protocols divided by the scope of questionnaire and physical examination, physical measurements, development evaluation and consultation, health education, and oral examination. Among these, the developmental evaluation is the major screening item of NHSPIC, with the first 9–12 months, the second in 18–24 months, the third in 30–36 months, the fourth in 42–48 months, the fifth in 54–60 months, and the sixth in 66–71 months of age, all but 4–6 months, the first NHSPIC period [
1].
The developmental screening tools used in the NHSPIC should be able to comprehensively assess the developmental domains to sensitively identify children at risk of suffering from neurodevelopmental disorders. As part of the NHSPIC project, Korean Developmental Screening Test for Infants and Children (K-DST) is a new developmental screening tool tailored as per the characteristics of the Korean children [
2-
4]. It was developed by experts in related fields including the ‘The Korean Pediatric Society,’ ‘Korean Society of Pediatric Rehabilitation and Developmental Medicine,’ ‘Korean Academy of Child and Adolescent Psychiatry,’ and ‘Korean Psychological Association.’ For pre-school children under 6 years of age (4–71 months), K-DST is a parent-reported screening test that allows parents to directly monitor the development of their children, taking less time and is able to quickly and effectively identify any developmental delays in primary care institutions. The tool was developed to allow the use of not only printed test sheets but also on-line test to increase the accessibility to examinees. This study reviewed the development of K-DST and evaluated the standardization and validation of this tool to see how accurately it could identify the target at risk of neurodevelopmental disorders. Moreover, we investigated the reliability and validity of the revised K-DST, which established new cutoff points by means of restandardization in 2017 based on the 3.06 million cases accumulated since the first edition of K-DST was conducted in 2014. This study was conducted after obtaining approvals from the Institutional Review Board of Korea University Guro Hospital (2013GR0048, 2016GR0083, 2017GR0223).
Discussion
Since the 2000s, the low birth rate coupled with an aging society has led to the widespread perception that social responsibility should be enhanced to create an environment favorable for childbirth and child nurturing. NHSPIC is in line with the basic direction of the nation's welfare policy, which aims to progress towards a better welfare state through investing efficiently in the health of the people.
One of the effective ways to assess development is to adopt standardized developmental tools. Developmental screening test is a test developed to screen children who do not lie within the normal range of development and determines whether professional diagnosis is required [
12]. The development of infants and children should be considered as a continuous series of functions that are variable. Therefore, developmental screening test should be conducted more than once, and it is essential to repeat the test in time series in order to monitor the developmental changes of infants and children with age [
13,
14].
The developmental testing tools used in Korea are either developed in-house or in foreign countries. The latter was used after some amendments and standardization by Korean standards or re-standardizing foreign tests. Some of the developmental tests developed in foreign countries include Denver Development Assessment Testing, Bayley Scales of Infant Development (BSID), Early Screening Inventory (ESI), and Developmental Indicators for Assessment of Learning (DIAL) [
9,
15-
17]. The developmental testing tools that were developed in foreign countries and standardized in Korea include Korean Denver-II, Korean DIAL-3, Korean Ages and Stages Questionnaire (K-ASQ), Korean Bayley Scales of Infant Development (K-BISD), and ESI-Revisited (ESI-R) [
9,
15-
18]. However, most test tools used in Korea are limited to children living in cities and thus not reflective of the national samples based on the standardization process. Also, some tests did not report the characteristics of samples such as areas or demographic information.
The NHSPIC, which has been implemented in Korea since 2007, utilizes developmental screening tools to evaluate development, but initially used either K-ASQ or Denver-II. However, Denver-II is difficult to apply to all children in the Korean medical environment, as it is time-consuming since the examiner must perform the test in person. Therefore, K-ASQ was favored as parents would fill out the questionnaire themselves in the NHSPIC. K-ASQ is a Korean version of Ages & Stage Questionnaires, the second edition developed in the United States and standardized for Korea [
19,
20]. However, K-ASQ is a test developed in the United States, which makes it unsuitable for Korean infants since children grow in culturally different environments.
To verify the standardization and validity of K-DST, the reliability analysis was conducted by evaluating the coefficient of internal consistency, construct validity analysis, sensitivity and specificity analysis, ROC curve analysis, and criterion-related validity analysis.
The results obtained by the analyses were as follows. In the item analysis identified by the IRT, the variability of most questions was higher than 1.7, which was very high by the criterion of Baker [
21]. The internal consistency coefficient for each domain for reliability analysis showed good Cronbach alpha values of about 0.73–0.93 in most domains.
Compared with K-ASQ, showing that the RMSEA value was significantly lower in the model's fit at 0.096–0.118 over 24 months, K-DST's Confirmatory Factor Analysis confirmed that the RMSEA value was within the acceptable range of 0.052–0.097 for all the age groups [
22]. Therefore, the structure of K-DST was verified.
The sensitivity of the K-ASQ was 0.75 and the specificity was 0.86 and that level was reportedly good [
18]. The sensitivity and specificity of the K-ASQ in the 30 and 36 months groups were 0.88–0.96 and the accuracy was 0.92 and 0.89, respectively, with good discrimination power in general. In contrast, in the 60 months group, the specificity was high at 0.95, but the sensitivity was relatively low at about 0.65 [
23]. In comparison, the results of the sensitivity and specificity analysis of the first edition of K-DST showed that the sensitivity was 0.886 and the specificity was 0.951, which is generally higher compared to that of K-ASQ.
In addition, The K-ASQ had the ability to select clinical groups such as intellectual disability and autism spectrum disorders as a risk group for neurodevelopmental disorders, but failed to select the groups with delayed language development such as developmental language disorders [
23]. In contrast, the ROC curve analysis results of the first edition K-DST showed an AUC value of 0.9 or higher for each disease. In particular, in the case of developmental language disorders, AUC values of 0.924 for language and 0.957 for social skills showed high correlation with the subdomains associated with neurodevelopmental disorders.
In the criterion-related validity analysis, K-ASQ showed statistically significant correlation with criterion variables but not generally high. There were also areas where there were no significant correlations related to other measures. For example, K-ASQ's gross motor domain in the 30-month age group had no significant correlation with K-BSID-II's motor quotient score [
23]. This suggests that it may be insufficient to predict the actual motor functions for that particular month. In comparison, the correlations between mental quotient of K-BSID-II and the cognition, language, and social skill domains areas of the first edition of K-DST showed high correlation at 0.60, 0.68, and 0.67, respectively, and the correlations between motor quotient of K-BSID-II and the gross motor and fine motor domains of the first edition K-DST were also high at about 0.57 and 0.61. Between K-WPPSI and first edition K-DST, the correlation between VIQ and the cognition and language domains of the first edition K-DST was higher than compared to that of the other developmental domains. In addition, the correlation between PIQ and the gross and fine motor domain was relatively high at 0.69 and 0.66, respectively. These results suggest that although K-DST is a developmental screening test based on parental reports, it has a high correlation with K-BSID-II and K-WPPSI, which are the most widely used test tools for confirming neurodevelopmental disorder. This suggests that K-DST is a highly reliable developmental screening tool for Korean infants and children.
In the case of revised K-DST, the sensitivity slightly decreased (from 0.847 to 0.833) in the newly adjusted cutoff point analysis compared to the existing cutoff point, while the specificity increased slightly (from 0.953 to 0.979). The false-negative values increased slightly from a value of 15.3% for the previous cutoff points to 16.7% using the new cutoff points, while the falsepositive values decreased from 4.7% for previous cutoff points to 2.1% for the new cutoff points, reducing the probability of judging a normal child as a child with developmental problems
According to the ROC curve analysis of the revised K-DST, the highest AUC values were found in the gross motor domain for cerebral palsy, the language domain for developmental language disorder, the social skills and language domains for autism spectrum disorder, and the cognition domain for intellectual disability. Compared with the first edition of K-DST, the revised K-DST showed increasing AUC values of the language and social domains (AUC=0.980, 0.998) in developmental language disorder, and also higher AUC values of the cognition, language, and social domains (AUC=0.960, 0.974, 0.979) in intellectual disability, showing better discrimination power for the revised K-DST compared to that of the first edition K-DST. In the correlation between K-BSID-II and revised K-DST, comparing with the first edition K-DST, the correlation between Mental Age Quotient and revised K-DST increased from 0.60 to 0.739 in the cognition domain, and from 0.68 to 0.766 in the language domain. The correlation between Motor Age Quotient and revised K-DST, also increased from 0.57 to 0.695 in the gross motor domain, and from 0.61 to 0.668 in the fine motor domain, signifying better criterion-related validity than that of the first edition K-DST. In the case of K-WPPSI-R, K-WPPSI-IV and revised K-DST, also showed an increase or slight decrease in the correlation between VIQ and the language domain from 0.74 to 0.701 in K-WPPSI-R, and from 0.74 to 0.770 in K-WPPSI-IV. Increased correlation between PIQ and the fine motor domain was also noted, increasing from 0.66 to 0.681 in K-WPPSI-R and from 0.66 to 0.700 in K-WPPSI-IV. Therefore, we verified that the revised K-DST has a higher criterion-related validity compared with that of the first edition of K-DST. However, the correlation between the gross motor domain of the revised K-DST and K-WPPSI-R and K-WPPSI-IV was somewhat low (r=0.513, 0.564). This is because the minimum age of the Wechsler intelligence test should be above 30 months, which is considered to be the main development period of the fine motor domain as opposed to that of the gross motor domain.
The illustration of the results derived from this study is as follows. First, the first edition K-DST showed high sensitivity and specificity compared to K-ASQ, which is known as a suitable screening test, which distinguishes well various neurodevelopmental diseases disorder. In particular, K-DST is an independently developed tool that suits the characteristics of infants in Korea, not a modified or standardized tool for existing foreign tests, and the reliability and validity of the tests were verified through various analyses. Compared to the K-BSID-II and K-WPPSI, the correlations for reliability and validity were high, therefore the validity as a screening test is confirmed. Second, the revised K-DST has become a more powerful discriminating tool because the AUC values are higher in the ROC curve analysis, and the correlation coefficient is higher in the criterion-related validity analysis as compared to that of the first edition of K-DST. Since the revised K-DST had the advantage of using 3.06 million NHSPIC’s big data and the distribution of the population sample is assumed to be asymptotic and the cutoff points were set using percentile scores, the revised K-DST exhibits higher reliability and validity as compared to that of 1st edition K-DST. However, in the case of revised K-DST, the number of certain age group, not included in the NHSPIC schedule were small (27–29 months). As a result, the cutoff point for the 27–29 months group is lower than that of the other age group, which limits the interpretation of this particular section.
In conclusion, K-DST is an independently developed tool that has been developed to suit the characteristics of Korean infants and children, in addition, it is a screening test tool that checks the reliability and validity through various procedures, and is restandardized using large-scale data. In particular, the revised K-DST has a higher sensitivity and specificity compared to K-ASQ, which is considered as a good screening test tool, and shows better discrimination ability compared to the first edition of K-DST through the ROC curve analysis and criterion-related validity analysis. The K-DST shortens the test time and enhances accessibility to the examinee by allowing the test to be conducted on-line as well as in paper-and-pencil test. K-DST can be utilized as a useful developmental screening tool for NHSPIC, as well as a developmental assessment tool for developmental surveillance, screening, and posttreatment changes of normal infants. In the future, if the standardization is conducted periodically using the accumulated data from NHSPIC, it can be applied as a better tool through the application of changes based on the sociocultural development period. As more clinical data accumulate, further research may be required to determine how the classification accuracy of K-DST varies in a more diverse group of neurodevelopmental disorders that were not included in the validation study at the time of development.