Can we trust the results?

No-one can guarantee that every answer given by every respondent in a survey is completely accurate and honest.  However, we can go a long way to improving our confidence in the figures by taking care over each aspect of the process:

  • Questionnaire design

We strive to ask questions in a clear and unbiased way, concentrating on what has actually happened in pupils' and students' lives.  So, we ask questions like 'what did you eat or drink before lessons this morning?', and not 'what do you usually eat for breakfast?' (which, in just seven words, makes three different assumptions that might not be correct!).  Questions are scrutinised before use, not just by SHEU staff, but also by expert groups from local authorities, such as teachers, advisers and public health professionals.

  • Piloting

Questions are tested with smaller groups before use across our services.  Once we have evidence that the answers from small-scale work are meaningful and accurate, we can support more wide-scale use.

  • Post-questionnaire interviews

We have regularly conducted interviews with pupils and students following completion of a pilot questionnaire.  These interviews can let us know that pupils and students are willing and able to find and choose an answer that properly reflects their situation.

  • Introduction to the pupils and students

Pupils and students must feel that the exercise has some value if they are to cooperate.  We produce extensive guidance for schools explaining ways of introducing the purpose and nature of the survey to pupils and students. 

  • Feedback from schools about data collection

We ask for feedback from every single class that completes one of our surveys.  Staff who supervise data collection are alert to problems that arise and provide important feedback on questions that could be worded better. 

  • Response checking

Staff at SHEU inspect every set of responses coming into the Unit.  Our experienced data preparation staff are sensitive to any odd-looking responses and we will respond, perhaps by excluding an individual's questionnaire (if they are not taking it seriously) or even by excluding everyone's answers if the responses from one question are not consistent with other trusted information.

  • Internal reliability

Questions in one part of the questionnaire can be related to another; for example, a question on recent spending might report spending on sweets, which can be compared with a question many pages away about eating habits.  When we do these comparisons, the match is always very good.

  • External reliability

Figures from one survey can be compared with results from another survey.  For example, we know a lot about patterns of cigarette smoking among young people over the last 30 years from Government surveys.  The questions they use have been double-checked, not just using the approaches described above, but also by saliva tests that show if someone is smoking.  SHEU figures show very much the same levels and also the same rises and falls over the years as are seen in the Government reports.

  • Feedback from schools and authorities

Once results are circulated, we invite feedback from schools and authorities by asking directly if they have confidence that the figures accurately reflect their young people.  They may have other supporting sources of information that we aren't aware of, or a sense that the picture they see is about right.

 

For more information, see the SHEU publications from the Young People into… series, or contact Dr.David Regis, the Research Manager at SHEU.

Topics: 

Comments about SHEU

Any comments on specific survey questions that may have caused difficulty?
Pupils at our primary school found the questionnaire very easy to understand and most of them completed the questions in less than 45min.

Teacher

"The Unit has a unique historical and contemporary archive of young people." Prof. Ted Wragg 1938-2005

Prof Ted Wragg, 1938-2005

"...the only question to cause a problem was 'has everyone got a pen?" Supervisor's notes following a school survey

Supervisor's Notes

"I very much value the contribution the Health Related Behaviour Survey has made to the public health agenda and feel confident it will continue to do so." Tribute from a Director of Public Health to John Balding, presented at his retirement lunch, May 2005

Director of Public Health

"As a result of the survey we reviewed and amended PSHE schemes of work, we are currently working on a "Green Travel Plan", a morning breakfast club was established and we further developed 6th Form mentoring."

Health Education Coordinator

"I really appreciate the professional service which SHEU offers.  We have had a great experience working with Angela on the school surveys." 

Health Improvement Specialist

"Many thanks for all of the fantastic information that you have sent to me over the years, it has really helped me to plan relevant courses for my students to follow and to help me to focus on the needs of the students I teach."

PSHE teacher

(Our) Senior team were very enthused with the rich source of data provided within the reports (and thought that the analyses including within the appendices section of the main reports were really interesting).

Health Improvement Specialist (Children, Schools and Families)

"Brilliant - thank you Angela. As always you and your team are so proficient at getting our requests dealt with so promptly - it is a real pleasure to work with such a great organisation."

Health Improvement Adviser

"The Unit is to be congratulated in preparing ... material of the highest standard and worthy of wide dissemination." National Association for Environmental Education

National Association for Environmental Education

"Thanks for presenting the survey to local schools this morning, I just wanted to thank you for such interesting and thought-provoking information.  
I’m really glad we were able to take part - the information (particularly headline data and differences) will support us to have some really interesting questions with the Year group as a whole about the sense they’re making of this; what they think it might mean in terms of changes they might make, and what they need to support them in this."

Deputy Headteacher

"I have valued greatly the work I have done with the team in Exeter, it has been a highlight of my years here." Health Promotion Specialist

Health Promotion Specialist

"Our use of the Health-Related Behaviour Questionnaire was commended as part of our accreditation for the National Healthy Schools Scheme." Headteacher

"Just to say a huge thank you for all your efforts in helping us with the Health survey amongst pupils. It has provided us with significant data which will be used across the school to help us improve. It helped us to obtain a healthy schools standard as well. I hope we can make this an annual feature as we can track the changing health of our pupils." Headteacher

Headteacher

"The data for (us) are very useful ... This is especially important when evaluating the impact of interventions regarding alcohol or other areas, as the survey data are likely to provide an earlier indication than routine data sources."

Specialist Registrar in Public Health

"We would like to take part in the next ECM survey. We have found the data produced invaluable for supporting evidence in our SEF etc."

School Vice Principal

"We're very happy to commission another survey from you. Our colleagues in School Improvement are dead keen to work with us on this. During our last LA Inspection, we were flagged from our Tellus data as having a bullying problem. We could demonstrate with our SHEU data - which had a much better sample size and coverage of the authority - that we did not have the problem they suggested. The Inspectors went away happy and we are definitely surveying again with SHEU."

Local Authority Senior Adviser

"We never make a move without looking at these excellent reports." Public Health Consultant

Public Health Consultant

Please could our College be part of the SHEU survey this academic year? We did it in 2016 and 2017 and found it incredibly informative and it helped us shape our PSHE provision.

Assistant Principal

...our analyst here in Public Health- is beside himself with excitement about all the juicy data pouring in...he can't wait to get his hands on it!!!!
He is happier than I have seen him for years.

Public Health Principal

"Your work in developing the Health Related Behaviour Survey was ground breaking and has continued to evolve." Tribute from a Director of Public Health to John Balding, presented at his retirement lunch, May 2005

"Many thanks for a major contribution to the health of children in the UK and elsewhere over many years and putting in place the continuation of the Unit." Tribute from a Director of Public Health to John Balding, presented at his retirement lunch, May 2005

Director of Public Health

We were all very impressed with the spreadsheet and can see that an incredible amount of work has gone into creating this!

Health Improvement Specialist

"You and the team have the evidence to show how young people's behaviour has or hasn't changed over time." 
Tribute from a Health Education Co-ordinator to John Balding, presented at his retirement lunch, May 2005

Health Education Co-ordinator

"Please send an additional copy of our report - it is the most requested and borrowed item in the whole library." Health Promotion Unit

Health Promotion Unit

"I really think that the HRBQ is a wonderful piece of work in terms of getting useful information for so many different organisations in one go." Healthy Children's Research and Statistics Officer

Research and Statistics Officer

"...the most comprehensive health education survey."

Daily Telegraph

"Every school involved in the National Healthy School programme should start with an HRBQ survey." Health Education Co-ordinator

Health Education Co-ordinator

"The Health Related Behaviour Survey is an incredibly useful resource for (us) as it provides schools, with invaluable data which can inform curriculum content, methods of lesson delivery and empower schools to better meet the needs of their pupils."

Health Education Advisor

"We were talking about (the SHEU survey) data at our recent NSCoPSE Conference, for PSHE advisers and consultants. It would be really helpful if some of this powerful data and the trends could be shared in the consultation around the PSHE Review. Colleagues shared their very positive experiences of (the SHEU survey). It provides excellent evidence of behaviour change for children and young people and of the impact of PSHE and wider interventions."

Personal and Social Development Consultant

"One year (following the SHEU survey) responses from our Year 4 cohort caused us concern, so we put in place a number of team building, motivational projects. We then assessed their effectiveness by requesting the SHEU questionnaires for these pupils as Year 5's."

Learning Mentor

"Many thanks to SHEU for your excellent professional support over the years."

PSHE teacher

"Within the curriculum, we are part of the Healthy Schools programme - and the local, Director of Public Health Award. We cover many facets of health from emotional intelligence to safety education and our very strong, Anti-Bullying and Child Protection programmes. You can imagine our delight when the Local Authority and our school nurse made the following comments after we took part in the regional Schools Health Education Unit Survey: " Head Teacher.
“This was an amazing set of outcomes and really good evidence that (your school) is doing a wonderful job in prioritizing the health and well-being of its pupils … Well done to staff, governors and parents for all your work on this through the Director of Public Health award and other strategies. It is very clear that pupils feel happy, safe and involved at the school and your caring ethos shines through this data.”
Healthy Schools Coorduinator.

 

Headteacher & Healthy Schools Coordinator

"You have made a truly significant contribution to health education and health promotion of young people in, not only England, but all over the United Kingdom and beyond." Colleague from NHS Scotland paying tribute to John Balding, presented at his retirement lunch, May 2005

NHS Scotland

"The children found the survey very interesting and enjoyed doing it." Class Teacher

Class teacher

"Thank you very much, David, for another excellent survey.  We look forward to receiving our reports."

Healthy Schools Co-ordinator

"The service you provide is of national significance." Health Development Agency

Health Development Agency

"This is an excellent way of keeping up to date belt and braces style."

School Drugs Advisor

"The Schools Health Education Unit has gained a reputation as one of the most reliable sources of information about young people's health." BBC

BBC

I just wanted to write and tell you how much I enjoyed your talk at last week's conference.

Chief Executive, substance use charity

"You have often stood alone against the media who were often looking for the sensational headline. I have noticed an important change: the media now look out for and report very fairly and fully on the reports from the team." 
Tributes from a Health Education Advisor to John Balding, presented at his retirement lunch, May 2005

Health Education Advisor

"Thank you from my staff to you and all your staff. The speed of 'turn-around' of the questionnaires is outstanding in anyone's terms." Headteacher

Headteacher

"Over the last twenty years you have achieved much. The surveys and subsequent reports have painted the clearest picture we have of what young people are doing and what they think." OFSTED 1998

OFSTED

I would be extremely interested to see the results as I know how useful this information has been to the other schools in the
borough

Headteacher

"We are planning next year's programmes around this information." Health Education Adviser

Health Education Adviser

"SHEU data proved the best source of the kind of information we were looking for (...) to provide research support to the National Healthy Schools Programme." 

Department of Health

"The Schools Health Education Unit has been a unique inspiration to all of us. For me, as I have worked in the many different areas of the NHS, the SHEU, its principles and your determination have always been a cornerstone in what a health promoting service should be about."
PCT Performance Manager paying tribute to John Balding, presented at his retirement lunch, May 2005

PCT Performance Manager

"I would like to say how much we appreciated the work you and your team have put in to this project, a big thank you for the excellent reports that you have completed on our behalf." Assistant Director of Public Health

Assistant Director of Public Health

"I would like to say that this survey was very useful and made me realise things about PE and health that I had never realised before......Food at school is groovy, especially if your school does Jamie Olivers School Dinners. Viva apples and thanks for the survey." Female pupil, 13 yrs old

Female pupil, 13yrs

"I have never looked at myself in this way before." Pupil

Pupil

"It's good to talk with you again - we used so much of those data and did so much with it in schools - we really got the place buzzing!"

Consultant in Public Health

"The (named) Children and Young People's Partnership has benefitted from the results of the SHEU survey locally for many years now, and we should like to continue to do so in future."

Consultant in Public Health Medicine

"You have made a fantastic contribution to children’s health education and promotion. I am personally grateful to you for helping to kickstart my research career." Prof. Neil Armstrong tribute to John Balding, presented at his retirement lunch, May 2005

Prof Neil Armstrong

"This week I have been working on a major strategy for service design -- it is easy to get solely focussed on hospitals, performance targets, and work force planning -- all very important; but at the same the needs of young people and how we need to work across agencies to address the health needs of today and tomorrow must be recognised. SHEU is founded to do just this."

PCT Performance Manager

"The Unit produces an invaluable body of knowledge... providing exceptional information across time and throughout the country." Kelloggs

Kelloggs

"On behalf of all the health promoters in Scotland I would like to say a big thank to you and your colleagues for your excellent work over the years. This includes not only your survey work but your role as a visiting examiner in Scotland and adviser on course development."
Tribute from a Health Commissioner to John Balding, presented at his retirement lunch, May 2005

Health Commissioner

"As a Deputy Head in a large secondary school I was involved in taking part in a city wide health and wellbeing survey over a period of six years. Completing the survey every two years grew in importance year on year, with the final cycle having a major impact on our SDP, PHSE curriculum, Ofsted outcomes and governor understanding.
Over the six year period we moved from a small sample in two tutor groups filling in a paper survey to two year groups completing an online survey. The reports produced give graphical analysis of a wide range of issues. As a result of the survey we increased the number of PSHE workshop days for students to address issues such as smoking, drug and alcohol awareness, anti-bullying workshops. The surveys helped Governors make a positive informed decision to allow Brook Advisory Clinic nurses on site to support students.
As a result of taking part and using the evidence provided we were able to offer more support for students which had a direct impact on improved attendance and outcomes."

Deputy Head Secondary School

"We have never consulted our young people like this before. The survey makes a great contribution to our 'best value' planning." Sports Development Officer

Sports Development Officer

"I would like to take this opportunity to thank you for your work regarding writing and compiling the sex education survey. The survey was well executed and the schools have found their individual reports very helpful. The results of the survey have enabled the Local Campaign Group to justify the need for young men's campaigns and given us invaluable insight as to the thoughts and experiences of this target group."

Teenage Pregnancy Strategy Manager

 “The (SHEU survey) helped us to prioritise where we needed to be in terms of PSHE education. We delivered assemblies based on the evidence as well as curriculum development, and dealt with whole school issues – particularly in regard to pastoral care. The answers received to the question on the survey “Who are you most likely to approach if you needed help” worried staff as “teacher” was not a popular answer. Subsequently the staff asked themselves why this had happened and what needed to be done to address the issue. There was more emphasis on wider aspects of PSHE education delivery, which needed more attention.

To summarise, the (SHEU survey) allows the PSHE department to assess the impact of teaching and learning and modify future lessons accordingly. It allows our school to look at whole school issues such as the extent to which the pastoral care system is meeting the needs of our pupils. It helps us to do need analysis of our pupils. It helps to provide important evidence for SEF / the extent to which we are meeting wellbeing indicators / National Healthy School standards.”  

Secondary School Head

"The Schools Health Education Unit is the jewel in the crown of the Health Education Authority." Major General Sir John Acland 1928 - 2006

Major General Sir John Acland

"The data from last time were spot-on and we have done lots of work with it. We are very keen to repeat the survey." Headteacher

Headteacher

“(The SHEU survey) was very, very useful. It gave us reassurance we weren’t missing a trick. For example not many pupils in the sample year groups were taking illegal drugs, which re-enforced our opinions. But the survey also raised issues and flagged some things up. We discovered that some of our girls weren’t eating enough – the percentage of girls in our school not eating lunch the day before the survey was higher than the county average. There were other concerns too, specifically around cigarettes, alcohol and attendance.
The school used this data and took a number of actions to address it. More female peer mentors were put in place and the school asked NEXUS (the Extended Schools service) for help, so they developed a programme for girls which addressed their eating patterns, healthy eating, sex education and self-esteem issues.
We ran an anti-bullying group for Year 9 as a preventative measure, based upon data provided by our current Year 10 students.
The travel data revealed that a high number of pupils took the car to school so we involved the BIKE-IT scheme who ran assemblies, brought in their bikes (including one with a pedal-powered smoothie maker!), and raised awareness of health and green issues.
The information about how happy the students were with their lives raised some concerns as far fewer girls were as happy as the boys, so work was done around developing aspirations, role-models and self-esteem."        
 

Deputy Head, Secondary School

"The system works and I find quite a lot of it useful in my work. I've also recommended it to others."

Teenage Pregnancy Manager

"The survey reports have been used to inform commissioning at specific commissioning groups. They are also being used within our Extended Schools Clusters and to inform The Annual Public Health and the Joint Strategic Needs Assessment." Programme Manager - Young People

"The credit goes to you for the fabulous information the survey yields!"

Assistant Director Schools and SEN

"We use the data to inform whole school practice: Pastoral programmes for target groups of pupils; Items for discussion with School Council; Information to help us achieve the Healthy School gold standard; To develop and dicuss with pupils our Anti-Bullying Policy; Targeted whole class sessions with the Police Community Support Officers; To share pupil perceptions of all aspects of their school life with parents, staff and governers." 

Learning Mentor

"Over the last twenty years you have achieved much. The surveys and subsequent reports have painted the clearest picture we have of what young people are doing and what they think." Tribute from OFSTED to John Balding, presented at his retirement lunch, May 2005

OFSTED