Speaking statistics: a short dictionary

Last month we discussed that learning the language of statistics is crucial to communicating your research. Now is the time to look a bit more closely at some ambiguous terms that are often thrown around in conversations on analysis of research data. Obviously, this is not an exhaustive list and if you would like to check out some other statistical dictionaries, the Oxford reference and the UC Berkeley Glossary of Statistical Terms are good starting points.

Kaplan et al. (2009) have published a table of 36 lexically ambiguous words, of which they discuss five in more detail in their paper. I randomly selected another 5 to put under the looking glass here.

Bias

In English, bias refers to the inclination or prejudice for or against one person or group, especially in a way considered to be unfair. In the context of dressmaking it means a direction diagonal to the weave of a fabric (e.g. a silk dress cut on the bias). 

In statistics, bias refers to a systematic as opposed to a random discrepancy between a statistic (estimated from the data) and the truth (expected in the population). So, put simply, a measurement procedure is said to be biased if, on average, it gives an answer that differs from the truth. Statistical bias can be unknown, unintended or deliberate. Whilst often undesired, there is no specific negative statistical connotation as there is in English.

Error

When somebody makes an error, it’s commonly understood as making a mistake. In statistics, an error (or residual) is not a mistake but rather a difference between a computed, estimated, or measured value and the accepted true, specified, or theoretically correct value. The error often contains useful information around distributional assumptions and model fit. 

We also often talk about Type I and Type II errors. These errors result from hypothesis testing where the hypothesis conclusion does not align with the underlying truth. In this context, the error is indeed a “mistake”. A Type I error occurs if the null hypothesis is rejected when in fact it is true (i.e. false positive), while a Type II error is not rejecting a false null hypothesis (i.e. false negative). 

Mode

The English dictionary defines mode as a manner of acting or doing, or a particular type or form of something. It can also be a designated condition or status, as for performing a task or responding to a problem (e.g. a machine in the automatic mode). In philosophy, a mode is an appearance, form or disposition taken by a thing, or by one of its essential properties or attributes. While in music, mode refers to any of various arrangements of the diatonic tones of an octave, differing from one another in the order of the whole and half steps.

In statistics the mode is a measure of central tendency and it is the value that appears most often in a set of data values. The numerical value of the mode is the same as that of the mean and median in a normal distribution, and it may be very different in highly skewed distributions. While the mean can only be calculated for continuous data, the mode applies to all data scales including nominal and ordinal data.

Null

Null means without value, effect, consequence or significance. In law, it refers to having no legal or binding force. In electronics, it is a point of minimum signal reception, as on a radio direction finder or other electronic meter. In mathematics, the word null is often associated with the concept of zero or the concept of nothing. And it is probably this association which drives the misconception that the statistical null hypothesis is by definition a test against 0. However, a null hypothesis is a type of conjecture used in statistics that proposes that there is no difference/relationship between certain characteristics of a population or data-generating process. 

Significant

Something significant in English indicates that it is sufficiently great or important to be worthy of attention. In research the term statistically significant is used when the null hypothesis is rejected with a sufficiently small p-value. The confusion arises when a statistically significant result is advertised as being significant (i.e. important) and meaning is attached to the size of the p-value. Lots of ink has flown over the pros and cons of this process and the misconceptions arising. But as a principle, you as a researcher will need to keep in mind that a statistically significant effect does not equal by definition an important effect. 

From experience when talking to clients, I would like to add two more words to this list that are often quite confusing because they indicate different things in different areas or even software.

Factor

In English a factor can be a circumstance, fact, or influence that contributes to a result. In mathematics t is also a number or quantity that when multiplied with another produces a given number or expression. In statistics a factor can take on different meanings depending on the context. In an experiment, the factor (also called an independent variable) is an explanatory variable manipulated by the experimenter. In a broader context, especially in software packages like SPSS and R, a factor refers to an independent categorical variable. In factor analysis, a factor is a latent (unmeasured) variable that expresses itself through its relationship with other measured variables. The purpose of factor analysis is to analyse patterns of response as a way of getting at this underlying factor. 

Covariate

Covariate does not necessarily have a specific meaning in English but in statistics two connotations are attached to it. Note that mathematically there is no distinction between the two interpretations. 

Covariates are often used to refer to continuous predictor variables. SPSS is particularly guilty of that. However, in its original sense a covariate is a control variable. And sometimes researchers use the term covariate to mean any control variable, including controlling for the effects of a categorical variable. In SPSS however, you would enter your categorical covariate as a fixed factor. Still following?

At the end of the day, lexical ambiguity can be resolved by being careful in setting up hypotheses, running analyses and interpreting results. This requires knowledge of the language as well as an understanding of the context in which you are applying statistics. Just like Rome wasn’t built in one day, these are just baby steps in the right direction. Through interaction with peers and experts, you will find your way and develop your statistical language.

References

Kaplan, J.J., Fisher, D.G., and Rogness, N.T. (2009). Lexical ambiguity in statistics: what do students know about the words association, average, confidence, random and spread? Journal of Statistics Education 17(3).

Marijke joined the Statistical Consulting Unit in May 2019. She is passionate about explaining statistics, especially to those who deem themselves not statistically gifted.

Random effects inference in linear mixed models: the good, the bad, and the misspecified

Francis Hui and Alan Welsh of the Research School of Finance, Actuarial Studies & Statistics at the Australian National University presented this talk to the Statistical Socity of Austrlia on Friday 25 September. Over 100 people showed up online, a tribute to the high regard in which the speakers are held and the importance of the topic they were presenting.

I was only able to attend the fisrt half of the talk, by Francis. Luckily for me he started at the end of his talk, with a summary slide. The upshot of it all was that for linear mixed models, if the random effects distribution is misspecified (i.e. not the usual Normal distribution which is typically assumed) then

* point estimation and inference for fixed effects only – is usually OK

* point estimation of variance components – is somewhat OK

* inference on variance components – be careful!

* prediction of random effects – is somewhat OK

* inference on the predictions of random effects – is not much known!

Francis and Alan are working on the last two list items. Today Francis showed the results of a collection of simulations for a simple mixed model, using six different named distributions for the random effects, ranging from t to Beta and chi-squared.

Assessing methodological quality in analytic research

Jennifer Stone is a PhD candidate in the Research School of Population Health at ANU. Her ANU supervisors were Professor Suhail Doi and Associate Professor Katie Glass. Jennifer gave her exit seminar on Thursday 24 September, the culmination of three years’ work, including time as an exchange student in the Netherlands studying preclinical animal systematic review methodology at Radboud University.

The core of Jennifer’s thesis was six papers on assessing the quality of papers that end up in meta-analyses. There are hundreds of quality assessment tools (over 400 in one of Jennifer’s papers!) and one of Jennifer’s contributions has been to propose a methodology to sythesize them all. She’s called it MASTER (Methodological Standards in Epidemiological Research). It contain seven sub-domains that address important aspets of quality such as recruitment, retenton and analysis. Another part of Jennifer’s project focused on the pervasive statistical issue of trade-offs between bias and variance (expressed as precision in this case).

The issue of stratification of meta-analyses by quality remains a conteted one and while Jennifer’s work suggests that it does not provide useful insight, it does still form part of Cochrane Collaboration recommendations.

Congratulations on reaching the reseach milestone Jennifer, and I hope your work is able to achieve impact across the research synthesis community!

Advances in Data Linking Methodology at the ABS

The Official Statistics Section of the SSA and the Australian Bureau of Statistics presented this webinar on Monday 21 September. Over 60 people joined online to hear Daniel’s presentation.

This talk represents a follow-up to Anders Holmberg’s presentation in mid-August. Unfortunately I was double-booked that day, but was pleased to discuover that this talk stood alone very nicely and was a clear and comprehensive view of the state of the art of data linking at the ABS.

There’s a bit of terminology to keep straight in your head first. Matches are what exist in the data, a record of a person over here in the Medicare database and over there in the Tax Office database. A link is what is made between them by the analyst, and it can be correct or not. The precision of a linking method is the same as the positive predictive value, and the recall of a linking methd is the same as the sensitivity.

Daniel described deterministic and probabilistic methods of matching, and the possibility that they may be equivalent. At the end of the talk he highighted several areas of current research including improvements n precicion, incorporating the uncertanty of a link in the standard errors of any linked data analysis, and the possibility of using an instrumental variables model.

Alice in Statisticsland: pearling a paper

“Five systematic reviews were identified investigating the effectiveness of [treatment] in [population]. These reviews were pearled to ensure all relevant studies were captured …”

Come again? The reviews were pearled? Yes, their citations were used to identify papers which have more citations which can be used to … and so on. The pearl of evidence grows inside the shell of the original research question. What a delightful image to brighten my day! Now I can tell my colleagues I sent the afternoon pearling and none of them will nned to picture me underwater cutting oyster shells off rocks!

Image source: Newcastle Herald

An application of network meta-analysis to evaluate the effectiveness of electronic cigarette on smoking cessation

The Statistical Society of Australia lunchtime webinar for Thursday 10 September attracted over 100 participants. The presenter was Dr Gary Chan, NHMRC Emerging Leadership Fellow in the National Centre for Youth Substance Use Research at the University of Queensland.

His approach to network meta-analysis is based n the papers by Rucker, Mills, Chaimani and Borenstein. You can look everything up on Gary’s github site.

His talk concentrated on the use of the netmeta package in R to carry out a network meta analysis of 11 studies of e-cigarettes, nicotine replacement therapy and placebos in terms of their effect on smoking cessation. The package has all the outputs you would expect from funnel plots to forest plots as well as nifty net heat maps to show which studies the estimated effect is being picked up from.

I thought the most ineresting question wasn’t the network related one ut the effect sze one, noting tht most studies of smoking cessation record missing outcomes as failures to cease. This can bias the effect size downward, and a sensitivity analysis or multiple imputation exercise around the percentage of such obervations could be a really important extension of the substance of the research.

There was also time for Gary to walk us through his GUI to R called StatsNotebook. Look out for its lauch in October-November this year!

Can I have some (more) please? Tips on writing qualitative research grant applications for quantitative researchers in population health

I was based in the ANU Research School of Population health for four yers, and I really enjoyed the research environment with its range from super-quantitative brain image scan models to super-qualitative studies of the lived experience of mental health care consumers. Professor Cathy Banwell sits on the quantitative end of this exciting specturm, and this seminar on Thursday 10 September represents the culmination of a number of months of conversations around how good it would be to expose quantitative researchers to qualitative ways of thinking, and vice versa.

Cathy was joined by Dr Christine LaBond, a qualitative research erin the School, to share her experiences and ideas as well.

Unfortunately I had to leave this talk early (for a second talk!). This happened just as Cathy was getting into the variety of frameworks within which qualitative research can sit, and the rich variety of qualitative data sources. Anything for photographs and videos to drawings and interviews can yield up valuable data to answer the relevant research questions.

The take-home message for me will have to be one of the points Cathy made right near the beginnng, that you can’t just say you’re going to record a few interviews and call it qualitative research, because there’s a whole framework supporting these techniques which grant writers ignore at their peril. True for qualitative research, and true for quantitative research too.

Complete sample likelihood analysis of complex surveys

Ahhh, sample likelihood! It hasn’t yielded up all its secrets, not even in the two decades since I carried out some research on this exact topic whilst on sabbatical at the University of Southampton. The seminar I gave on my return to Canberra was called “Round the World with Maximum Likelihood”. The likelihood of another round-the-world trip is pretty low at the moment, but opportunities to engage with the concept are clearly still plentiful.

Associate Professor Robert Clark of the Research School of Finance, Actuarial Studies as and Statistics gave this seminar to the School via Zoom on Thursday 3 September. He’s been working on this topic with Dr Francis Hui, also of RSFAS.

Robert put his talk in  the context of regression analysis of complex survey data. Why is this a thing? Why can’t you just request a regresssion in your avourite stats package, declare a vector of weights, and be done with it? Well you can do that, it’s called maximum pseudo-likelihood (Skinner et al 1989), and it’s a pretty common solution. However it ignores the extra layers of relationship in the sample design and so more nuanced solutions are definitely desirable.

Robert particularly introduced the complete sample likelihood approach, inspired by the maximum sample likelihood models of Pfeffermann et al (1998). He (or rather Francis) fitted these model using the TMB package in R. Model checking includes the use of the randomised quantile residuals published by Dunn & Smyth (1996). The complete sample likelihood estimates should eventually be as easy to implement as the pseudo-likelihood ones, with the benefit of smaller standard errors.

Health inequality in China with its rapid socioeconomic and demographic change

The School of Demography at ANU has moved its seminars onto Zoom and so I joined a couple of dozen staff and stuents online on Tuesday 1 September for this presentation by third year PhD student Ms Mengxue Chen.

She presented the three main blocks of work that constitute her PhD research. She’ using three main data sources: the Chinese Disease Surveillance Points 2006-2016, the Chinese Longitudinal Healthy Longevity Survey 2005-2014, and the China Health and Retirement Longitudinal Study 2005-2018. This last data set is very reminiscent of the Austalian 45 And Up study, with approximately 23,000 individuals starting in middle age.

The first two blocks of work that Mengxue described used more demographic techniques to produce lifespan disparity graphics for subpopulations such as male-female and rural-urban. These curious graphics have reversed axes to ensure that movement up the graph refers to positive health outcomes. I was wondering if it was also possible to put measures of uncertainty on to these point estimates to enhance the graphic further.

The final block of work Mengxue will undertake will consist of a three-level multilevel model to study the social determinants of health that can be estimated from the CLHLS and CHaRLS data sets. I’m looking forward to the resuts of this major analysis, and I wonder if there will be issues with missing data or loss to follow-up that can arise in large-scale longitudinal datasets. One of the other questions for Mengxue to consider was about the effects of internal migration (rural to urban, for example) that might lead, for instance, to the neessity of using time-varying covariates. It’s going to be an exciting year for Mengxue! Congratulations on a great seminar.