# National Visitor Survey methodology

The National Visitor Survey (NVS) is a large-scale telephone survey which has been conducted continuously since January 1998, and is designed to measure domestic and outbound travel by Australian residents. The NVS is funded by the Commonwealth Government, and state and territory governments under the auspices of the Australian Standing Committee on Tourism (ASCOT).

NVS concepts and definitions are based on those developed by the United Nations World Tourism Organization (UNWTO).

https://unstats.un.org/unsd/publication/Seriesm/SeriesM_83rev1e.pdf#page=36

Results from the NVS are published quarterly and are available for free from the Tourism Research Australia website, go to this page.

### OVERVIEW

NVS eligibility includes any Australian resident who is 15 years of age or more, and has lived in their current residence for at least three months.

NVS interviews are distributed evenly across most days of the year (with the exception of 12 public holidays), which means that the current annual quota of 120,000 interviews are completed at a steady rate of around 2,300 per week.

Sample size and sample type across the time-series:

- 80,000 annual landline sample from 1998 through to 2004
- 120,000 annual landline sample from 2005 through to 2013
- 120,000 annual dual-frame sample continues from 2014 onwards, but with a 50:50 mobile/landline split through to the end of 2017
*TRA plan to move to 60:40 mobile/landline sample split in 2018. Future increases in mobile sample share are likely, assuming the growth of mobile-only households continues.*

The types of trips collected by the NVS include overnight trips, day trips and outbound (international) trips, though some types of routine trips (such as same-day journeys to work) are excluded. Overnight or outbound trips of more than one calendar year in duration are also excluded.

To ensure each respondent is able to accurately recall the details of any trips, information is only collected for recent trips. The respective **recall periods** are seven days for day trips, 28 days for domestic overnight trips, and three months for outbound trips.

### DATA COLLECTED

The information collected from all respondents (whether or not they have reported recent travel) includes:

- place of residence
- respondent’s age and gender, marital status and employment status
- Household characteristics such as income, number of persons, etc.

For those respondents who have reported recent travel, information is collected for each reported trip, and some of the main data collected includes:

- main destination location for day trips and outbound trips
- stopover location(s) for domestic overnight trips
- purpose of trip or stopover
- accommodation and transport used
- leisure activities
- itemised trip expenditure
- travel party

The above are indicative of the type of information routinely collected in the NVS, however, there are more data items collected. Further, the questionnaires are modified each calendar year to collect information on emerging issues in the tourism industry, and to meet the information needs of the Commonwealth Government and the state and territory governments.

### INTRODUCTION OF DUAL-FRAME SAMPLE IN 2014

Up to the end of 2013, the NVS sample only included residential landline phones. From January 2014 onwards, the sample design was modified by the addition of mobile phones. This change was in response to the growing population of persons who live in mobile-only households. The population of Australians who do not have a landline phone is growing each year, and in 2016 was estimated to include more than 30% of persons aged 15 years or more.

Without the addition of mobile phone sampling in 2014, the NVS would have suffered from an ongoing increase in **coverage bias** which would have made the NVS estimates increasingly inaccurate over time.

Because many people have a mobile phone and a residential landline phone, there are effectively three phone ownership populations:

- persons with a landline phone only
- an “overlap” population of persons who have both phone types (and may be sampled by either mobile or landline)
- persons with a mobile phone only.

The NVS collects the same demographic and travel information from the landline and mobile phone respondents. Additional screening questions are asked of mobile phone respondents to ensure it is safe for them to be interviewed and that they are in-scope for the survey.

As the NVS now includes both mobile and landline sampling, it is referred to as an **overlapping dual-frame** (ODF) survey

**Figure 1.**Outline of an overlapping dual-frame telephone survey

### BREAK IN SERIES FROM 2014

As was expected, the travel patterns reported by people interviewed on mobile phones differ from those interviewed on residential landlines. The scale of those differences meant that **the introduction of mobile phone sampling for the NVS generated a break in series from the beginning of 2014**.

Due to differing geographic and demographic patterns of phone ownership, the break in series is more pronounced in some of the smaller states and territories, and the scale of the break also varies with trip characteristics such as purpose, average spend etc.

The NVS continues to have an annual sample of 120,000. For 2014 through to the end of 2017, the sample remains an even 50:50 split between mobile phones and landlines. Because the population of Australians without landline phone is increasing, it is anticipated that the sample share will change to a 60:40 mobile/landline split in the near future.

### NVS WEIGHTING

Trip weights for the NVS are calculated on an individual trip basis. The process takes into account the age group, gender and place of residence of the respondent, the size of the household in which they live (for landline sample), the month of travel, the recall period applicable to the trip, and the number of trips reported. As part of the calculation, groups of NVS trip records are benchmarked to the ABS estimated resident populations according to their age group, gender and place of residence.

The **overlapping dual-frame** (ODF) methodology does not present any particular challenges in terms of the sample design, however, as with any ODF survey the weighting process must be specifically structured to eliminate double-counting of the overlap population.

The weighting calculation is different for each of the three phone populations. In addition, the two halves of the overlap population (sampled by landline or mobile) are also separately weighted.

### REVISIONS TO NVS DUAL-FRAME ESTIMATES FOR 2014 AND 2015

#### Background

From the beginning of 2014, the NVS moved to an overlapping dual-frame methodology, which means that previous landline phone sampling was replaced with a mix of mobile and landline sampling.

Starting late 2015, the NVS began to show larger than expected increases in the estimated overnight trips and spend in some of the smaller states and territories, and in some tourism regions. The increases were most noticeable for trips relating to business travel.

Analysis of the NVS data indicated that many earlier figures (2014 and 2015) were under-estimated, which resulted in apparent high growth rates when these were compared to later time periods.

#### Source of the anomalies

TRA uses a range of ‘big data’ sources for the sample design and weighting of surveys results. This includes a dataset of Australian landline and mobile phone numbers for the design of the sample and call allocation for the NVS.

The mobile phone dataset available in 2014 did not have a good correlation between phone ownership and the phone ownership population by telco. In particular, the Telstra segment of the phone population was under-represented in 2014 and 2015 (See Table 1). Therefore, travel characteristics of Telstra mobile phone owners were under-represented in the domestic tourism estimates.

In December 2015, an improved mobile phone dataset became available and was added to the NVS sample design process. This dataset delivered improved estimates for domestic visitation due to better representation of the mobile phone population by telco. It was expected that a one to three per cent change would occur at the national level following this improvement.

Analysis of the NVS data indicated that earlier figures (2014 and 2015) were under-estimated, which had inflated growth rates particularly between 2015 and 2016. This was related to the implementation of the improved dataset of mobile phone numbers used in the design of the NVS sample frame.

Year | Telstra | Optus | Vodafone |

2014 | 35% | 35% | 30% |

2015 | 36% | 36% | 29% |

2016 | 48% | 35% | 18% |

**Table 1.**NVS sample distribution by mobile network provider

#### Corrective actions

TRA has re-estimated the NVS overnight data for 2014 and 2015, with the primary objective being to maintain the NVS time series. As the 2016 data was relatively unaffected, the back cast impacted mainly the 2014 and 2015 estimates.

The approximate modelling process:

- analysed the NVS data to understand the movements in various demographic metrics (including the telco shares) over the period 2014 through 2016
- re-weighted the data to repair the demographic metrics in 2014 and 2015.

Additional work involved:

- measuring the long-term average growth rate in various NVS estimates from 2005 to 2016
- estimating the 2014 and 2015 error in the volume estimates of
- visitors, nights and regional expenditure
- by state and territory, tourism region, purpose and interstate/intrastate.

- applying corrections to those volume estimates
- using ABS state-level economic statistics to fine-tune the overall NVS growth for the period 2014 through 2016. This was done for each state and territory.

#### Results

Revised estimates at the national level have seen changes in volume for trips, nights and expenditure of between one per cent and four per cent in 2014 and 2015. There is minimal impact on the 2016 estimates (See Figures 2, 3 and 4).

Overall, the average growth over the last five years has seen little impact. However, results differ across the various states and tourism regions dependent upon the mix of travellers that visit each of these destinations.

To see how the change impacts each state and tourism region, please use this interactive map and charts.

**Figure 2.**Domestic overnight expenditure, old and new calendar years

**Figure 3.**Domestic overnight trips, old and new, calendar years

**Figure 4.**Domestic nights, old and new, calendar years

### RELIABILITY OF NVS DATA

The results given in the NVS are based on a sample, rather than a census, of Australian residents. As with all sample surveys, the results are subject to sampling variability and therefore may differ from figures that would have been obtained if the entire Australian population had been included in the survey.

A measure of the possible degree of difference is given by the confidence interval, which indicates the extent to which an estimate might vary by chance from the true figure because only a sample of the population was included.

In Table 2, the 95% confidence interval widths are given for a range of estimates available in the NVS. These confidence intervals are expressed as a percentage of the estimate. That is, there are approximately 19 chances in 20 that the true figure (which might be obtained from a census) is within the range identified by applying the figures in the table.

**Table 2.**Confidence intervals for NVS estimates at the 95% level

In Table 2, the shaded areas indicate estimates that have large confidence intervals (greater than 50%, or greater than 100% of the estimate). These estimates are subject to sampling variability which is too high for practical purposes and should be used with caution in analysis or in the reporting of NVS estimates, as they have a large margin of error. All other estimates have smaller confidence intervals, i.e. the estimates are closer to the values that would be obtained if the entire Australian population were interviewed.

The following example illustrates the use of the table to determine 95% confidence intervals for NVS estimates.

The estimated number of overnight visitors to a particular state was 7,000,000. Looking at the ‘Overnight visitors’ column, this estimate has a 95% confidence interval of plus or minus 4.2%. Therefore, there are 19 chances in 20 that, if the entire population had been included in the survey, we would obtain a figure which is within the range 7,000,000 plus or minus 4.2% of this estimate, that is, in the range 6,706,000 to 7,294,000.

For users who are familiar with statistical estimation techniques and the use of standard errors, the values in the above table are derived from the following linear regression equation:

Where RSE is the *relative standard error* and the model parameters are:

In the above example of overnight visitors to a particular state:

The 95% confidence interval is then found by multiplying the RSE by 1.96

Although confidence intervals are useful in indicating the reliability of one data item, they cannot simply be added to detect whether movements in data items between two periods of time are statistically significant. The following formula gives an approximation of the confidence interval width for the difference between two estimates:

Note that this formula is an approximation only and assumes that the two estimates being tested are of similar size and are based on similar samples.

In the example used above, an observed change from our estimate of overnight visitors of 7,000,000 would be significant (at the 95% level), if there is a difference in a subsequent or earlier estimate of 5.9% (1.4142*4.2%) or more. Therefore, if another estimate is outside the range 6,587,000 to 7,413,000 (+/- 5.9%), we can state that there are 19 chances out of 20 that the apparent movement reflects a true trend in the population.

Use of NVS data to analyse national, state/territory, or regional domestic tourism performance should be based on an understanding of the data’s level of reliability. Through understanding the confidence interval of the data, users are able to determine when a change between years is likely to be statistically significant and when it is not (that is, when it is more likely to simply be the result of random sample variation).

#### Items that are not collected for long trips

In the NVS, information on purpose, accommodation, transport, leisure activities and detailed expenditure items are not asked for individual locations visited on long trips. A long trip is defined as one where a person stops overnight in more than 21 different locations.

#### Row and column totals

Items within the body of each table may not add exactly to row or column totals. This is due to rounding and to inclusion in totals of unallocated ‘not stated’ or ‘unspecified’ responses.

In some tables, the row or column total may be considerably different to the sum of the component values. This occurs due to multiple responses to some questions in the NVS questionnaire. Where items within the body of a table do not add exactly to row or column totals, a footnote has been provided.