Introduction

On June 7th we released a white paper on our research into ACE scores among adult Third Culture Kids with experiences of international mobility. Prior to the release of this more detailed report, we released a methodology report which explained our research practices, definitions, and motivations. This report outlines procedures followed and controls put in place to assure the quality of the data we gathered.

Background

TCK Training Founder and CEO, Lauren Wells, recognized the need for increased research as a part of the preventive care methods TCK Training was developing. Lauren’s three books on caring for TCKs hypothesized the prevalence of ACE scores and developmental traumas in the TCK community, while advocating for practical care methods to mitigate the potentially negative outcomes of these traumas. As TCK Training’s work grew to serve hundreds of families, organizations, and companies, it became even more apparent that certain data was missing from global TCK research, and that research-based approaches could not be built without knowing more information. She reached out to Tanya Crossman in early 2021 suggesting a collaboration.

TCK Training Director of Research and Education Services, Tanya Crossman, completed a TCK-themed masters research thesis through the Australian College of Theology in 2017 which was awarded a High Distinction (the highest possible grade in the Australian academic system).[2] The project utilized both quantitative and qualitative research, and required multi-level ethics approval. During her course of study, Tanya’s book on Third Culture Kids was released, which also included original qualitative and quantitative research.[3] Tanya leaned on these experiences during the creation and administration of the 2021 TCK Training survey.

Survey Parameters

The primary objective of the survey was to obtain quantitative data on Adverse Childhood Experiences (ACEs) among Third Culture Kids with experiences of international mobility that could be compared to existing research into ACEs. 

Third Culture Kids are individuals who spent some of their developmental years living outside the country where they have legal citizenship, usually as a result of mobility due to parents’ choice of work or study, with the intent for eventual repatriation. We chose this limited definition of Third Culture Kids who had experienced international mobility for several reasons. First, while we would love to provide resources for all Cross Cultural Kids (and hope this survey will be the first of many research projects to be undertaken by TCK Training) choosing to hold ourselves to a limited scope with clearly defined parameters was important for delivering clear research with replicable results. 

Second, both Lauren and Tanya have seen the impact of international mobility on Third Culture Kids, especially when seeking support from mental health professionals regarding childhood trauma. Many TCKs have had their trauma dismissed, diminished, ignored, or outright rejected, as their international mobility is seen as a privilege and not a source of trauma. Lauren personally was told by a therapist that although she was exhibiting symptoms commonly associated with PTSD, that couldn’t be the diagnosis because it sounded like her life had been full of exciting globetrotting experiences. Therefore, one objective of the research was to determine if there was a connection between international mobility and childhood trauma in a population perceived as privileged.

Third, international mobility also brings up issues of legalities and social welfare safety nets. While some countries assume responsibility for the welfare of non-citizen minors resident within their borders, others do not. A child who experiences abuse while living outside their passport country, for example, may have fewer avenues for support, not know what those avenues are, or not have access to them due to language or cultural barriers. They may be unable to escape abusive parents when they are not permitted to cross the border without parental permission. We are also aware of various instances in international communities where sexual predators were allowed to continue to move from place to place and have access to minors as adequate background checks were not conducted, quiet resignations were preferred to loud dismissals, along with other factors made easier by moving between jurisdictions and different sets of regulations. 

Through our work with internationally mobile families, we (both Lauren and Tanya) were anecdotally aware of these and other potential dangers. Many parents, however, see international communities as safe havens for their children. We hypothesized that the data we collected on ACE scores and developmental trauma would show that TCKs experiencing international mobility were exposed to the same if not more stressors and ‘dangers’ than people who grew up in a single home country.

Drafting the Questions

We curated a series of questions to mirror the language of existing ACE score questionnaires.[4] It was important that our ACE factor questions be similar enough to questions asked in existing ACE factor studies that we would be able to compare our results. This includes the subjective nature of childhood perceptions of safety. The question on emotional neglect, for example, takes into account whether a child feels loved and supported, even if in hindsight they know their parents loved them.

We made some small adjustments to the questions, but most of our adjustments were in the instructions, to accommodate the different living situations of many TCKs. For example, many TCKs experience having live-in staff at some point during childhood. This might include nannies, housekeepers, or even drivers that live in their family home and take on caregiving responsibilities. They are household members with significant influence in the lives of TCKs, even if they are not related. While these household members are not specified in other ACE questionnaires, we decided to include them because of their prevalence in the TCK context.

The one key change we made was the inclusion of a question regarding violence toward not only a child’s mother figure, but also toward a child’s father figure. (Some other more recent ACE surveys have made this same change.)[5] This change made little difference to the end results but we thought it was an important matter of recognition and inclusion. It resulted in an additional ACE factor for only 35 of the 1,904 respondents in our survey results, or 1.8% of the total respondents. If looking at a segment of 20% of the survey (such as a group with ACE scores of 4 or more) that additional 1.8% would result in a change from 20% to 20.36%. 

In addition to asking the ACE factor questions, we asked a series of demographic questions, including questions regarding mobility (both domestic and international), to enable us to sort the data and compare different groups. In the data analysis phase this has allowed us to control for particular factors driving higher or lower scores.

We also added a series of questions asking about additional types of developmental trauma. While we had a clear hypothesis and research goal with the ACE factors and demographics, in this area we were casting a wider net, and working from a place of openness and curiosity. We were interested to hear from TCKs about what they had experienced and witnessed first hand as children. This was not an exhaustive section, but it creates a starting place to see if additional research is warranted and what directions that might take.

Finally, we added an open-ended answer section at the end. This was an optional place for respondents to add anything additional they felt they had not been able to express through the preceding questions. Trends in these responses shed light on the overall data and potential future research avenues.

Expert Review 

After drafting our questions, we reached out to key experts in relevant fields for their feedback on our survey. We shared information about the goals of our research, and gave them access to the question list and instructions we had drafted. 

These experts included two academic researchers (both PhDs), a child psychologist, several therapists working primarily with expatriates and TCKs (psychologists and counselors), international school educators, and international education consultants. In lieu of an academic ethics committee, this group of expert advisors served to ensure we treated our participants with care - especially with regard to their emotional welfare given the difficult subject matter we were asking them to engage with.

In consultation with them, and in response to their feedback, we made significant changes to many aspects of our questions and instructions, specifically in the wording of sensitive questions and clarity of speech. 

We also created better and more clear content warnings so that anyone proceeding with the survey would know what to expect, along with more comprehensive resource lists, to provide easy access to helpful resources should answering the survey bring up difficult feelings they wanted help to process. Our expert advisors contributed suggestions to improve and add to these lists. 

Beta Testing

As an extension of our peer review, we provided a link to a beta version of the survey to a number of Adult TCKs around the world for testing purposes. (This included some of our expert advisors). These were people from very different backgrounds, to help test the margins of the survey and find places where it did not work correctly or as intended. We specifically invited both Adult TCKs who had experienced significant trauma and those who reported having a mostly positive TCK experience. In response to their feedback, which included several helpful suggestions, additional improvements to the survey were made. 

None of the beta test responses were included in the final research, as the questions they answered were not the final version, nor were they entirely anonymous in many cases. Beta testers were invited to retake the survey after the final version was launched.

Survey Launch and Promotion

After three months of review and testing, the TCK Training survey on Developmental Trauma in TCKs was launched online in late June 2021. Our intent was to collect at least 1,000 complete responses that fit our study parameters. We planned to leave the survey open while recruiting respondents from various demographics for six months. 

The survey was promoted through multiple online channels. We used email, social media, and virtual groups to publicize the survey among various demographics. We put information and regular updates out through our own networks (our newsletter, social media channels, etc) and contacted influencers with wide networks in different types of TCK-related communities. 

We connected with adult TCKs of various ages and demographics, asking for their support to share the information with their own networks. We also connected with a number of academics, including those who had been involved in peer review and beta testing of the survey. 

We created promotional images and information specific to the mission, military, corporate, diplomatic, NGO/IGO, and international school teacher communities, in an attempt to reach out to each of these specific demographics. This included sharing these graphics with particular individuals and communities. This approach was particularly effective at boosting engagement from the foreign services communities (both military and diplomatic). 

We also created graphics showing the demographics reflected in responses so far, and graphics noting demographics we were still hoping to hear from. This approach was particularly effective in achieving a wide range of ages: our respondents ranged in age from 18 to 89, with a reasonably even distribution.

The survey closed as planned at the end of December 2021. In total we received 2,377 responses. After applying the exclusion criteria outlined below, we accepted 1,904 of these responses for inclusion in our data set.

Exclusion Criteria and Data Quality

A number of parameters were placed around the survey responses to ensure quality in the resulting data set. 

First, we only accepted responses from adults - those who were at least 18 years old at the time they completed the survey. An exclusion clause at the start of the survey checked if a participant was over 18; if not, it would navigate out of the survey. In addition we also used the demographic question about year of birth to filter those who entered a birth year indicating they were in fact under 18. This extra step captured 35 under-age responses for exclusion from the data set.

Second, we only used complete responses in our final data set. We did not take into consideration whether or not optional questions (such as open-ended comments) were completed, but did require that every question related to demographics, ACE factors, and developmental trauma be complete. This means that all statistics are the result of an equal number of contributors. This eliminated 419 responses from our data set. (This includes responses where only one or two questions were answered before the survey was abandoned.)

Third, to maintain our narrow definition of Third Culture Kids who experienced international mobility, we used several methods to check our responses. There was an exclusion clause at the start of the survey where people could self identify as not belonging to our definition, and be taken to a disqualification page. We also used demographic questions to sort for qualifying responses. 

37 complete responses where a person said they had not lived outside their passport country before age 18 were individually reviewed. 19 were removed from the data set; 18 were kept. Many of those retained had lived in more than one country, some of them indicating that they had citizenship in each. This fulfilled the requirement of international mobility. In other cases their following responses contradicted the answer of 0 years outside their passport country, and it seemed more likely they intended to say they had lived 0 years inside their passport country. This shows why individual review is important in these cases. Getting a full picture of the demographics often made it clear whether a respondent did or did not match our survey parameters.

Next, the data set was checked for repeat answers: the same set of answers in more than one response. No two responses were identical. This left us with a final data set of 1,904 responses.

Finally, every response which did not fit the general categories we would use for our data analysis was individually reviewed. For both the sector and education categories, there was a “none of the above” option available should the respondent feel that their experience did not connect with any of the options we outlined. 

In the vast majority of responses reviewed, answers in other sections of the survey made it clear that their experience did in fact fit into our intention for one of those categories. In most cases, the problem was that we had not made the description inclusive enough for them to see themselves in it. In these cases, the responses were assigned to the appropriate category so that they would be counted in the statistics to be created during data analysis.

Common re-assignments included:

• Department of Defense schools as a sub-type of International School

• Military Base schools as a sub-type of International School

• Missionary schools as a sub-type of Christian International School

• Government aid organizations as a sub-type of the NGO/Humanitarian sector

• Entrepreneur parents already included in the Business sector

• “Moved for work” already included in the Business sector

Risks to Data Quality

We believe it is highly unlikely that anyone unknowingly answered the survey more than once. There were no identical responses, and a quality assurance measure built into the survey did not allow the same IP address to answer the survey more than once (it would instead offer the option to change your answers).

It is possible that someone could choose to enter false answers maliciously, but looking at the facts we believe the risk of this having happened is quite small. 

We did not require that individuals leave long-form comments, but 21% chose to leave a comment. In addition, while we also did not require individuals to leave an email address, 38% of respondents chose to share their email address. All told, 44% of total respondents left a comment and/or an email address. This is a significant portion when these actions were entirely optional.

Moreover, if we look at the ACE Scores of only the 719 respondents who left their email addresses, we see much higher ACE scores than in the overall survey results. Only 19.7% had an ACE score of 0, compared to 26.5% in the wider TCK group, and 36.1% in the CDC-Kaiser study of 17,000 Americans; 26.4% have a score of 4+, compared to 20.4% in the wider TCK group, and only 12.5% in the CDC-Kaiser study.[6] Additional information about data analysis and resulting statistics will be provided in our forthcoming white paper. This demonstrates that if there were in fact any malicious data in the unidentified responses, it served to drive down ACE scores in the group. In this case, looking at the whole body of data gives a more balanced perspective of experiences among the TCK community.

The other potential risk to data quality is human error in data pruning and analysis. The ACE scores were calculated manually by calling up the answers to the 10 ACE factor questions for each individual response and counting the number of ‘yes’ responses. Each individually calculated response was reviewed to check the number against their answers. This mitigates the possibilities of human error miscalculations. Additional random reviews have taken place at various times; no miscalculations have been identified during random sampling and review.

Conclusion

While the 2021 TCK Training Survey of Developmental Trauma in Third Culture Kids (TCKs) was not conducted through an academic institution, we trust that this methodology report shows the expertise, care, and attention to quality that has gone into its construction and administration, beginning months before the first responses were collected. More information about our data analysis, statistics about the ACE scores of TCKs, and comparisons between our data and existing ACE research, will be presented in our forthcoming white paper. 

Tanya Crossman & Lauren Wells
TCK Training
April 2022