Table 6 Process evaluation of interventions
Trial name: First author publication year (year study commenced), Country (Reference #) | ‘Process evaluation’ or satisfaction measurement | Evaluation components | Quantitative evaluation (including during or post-intervention) | Qualitative evaluation (including during or post-intervention) | Participant perceptions where evaluated |
|---|---|---|---|---|---|
Process evaluation through response on a 4-point scale - quantitative | At each session, participants were asked to rate usefulness and relevance of the program on a 4-point scale from “not at all useful/relevant” to “very useful/relevant.” (i.e., “How useful was the session overall?” and “How relevant was this session to you and your family?”) | Participants were asked to complete forms after each session and indicate usefulness and relevance of the program and components of each of the sessions (during intervention) | Qualitative interviews were conducted 3–5 months after the completion of the program (post-intervention) | • Preference for combination of delivery modes • Appealed to first-time mothers • Participants’ lack of time to participate due to return to work | |
Carlsen 2013 (2010), Denmark [29] | None specified | None specified | Participants were not asked to rate satisfaction | Participants were not interviewed | Not evaluated |
PRIMROSE: Doring 2014 (2014), Sweden [30] | None specified | None specified | Participants were not asked to rate satisfaction | Participants were not interviewed | Not evaluated |
Franco-Antonio 2018 (2018), Spain [31] | None specified | None specified | None specified | None specified | Not evaluated |
MumBubConnect: Gallegos 2014 (2010), Australia [32] | Process evaluation | Frequency of text messages sent and responses received; number of telephone calls made by breastfeeding counsellor; Qualitative responses gathered via questionnaire to obtain women’s acceptability of service | Frequency of text messages sent and responses received; number of telephone calls made by breastfeeding counsellor (during intervention) | Qualitative responses gathered via post-intervention survey questionnaire to obtain women’s acceptability of service (post-intervention) | • Considered themselves well supported through participation in program |
Gibby 2019 (2019), Hawai’I and Puerto Rico [33] | Satisfaction | Usefulness of text messages; how participants were influenced to change behaviours; and most and least liked messages; Satisfaction with the text messages delivered. Qualitative interviews at follow-up visits. | Most liked and least liked messages were rated by participants | At the follow-up visits, participants completed a qualitative interview regarding helpfulness of messages, ways in which receiving the messages influenced or changed feeding practices and overall feedback about receiving the messages. Responses to 6 open-ended questions (post-intervention) | • More likely to make changes if the content delivered aligned with their pre-existing beliefs • Level of engagement with programs fluctuated, based on their needs at a particular point in time and based on their child’s stage of development |
Steps to Growing Up Healthy: Gorin 2014 (2014), USA [34] | Process evaluation stated in study protocol paper, no evidence of one being conducted | At the end of intervention period mothers asked to evaluate the program - helpfulness, components most useful, refer friend to program | Not evaluated | ||
Healthy Habits, Happy Homes: Haines 2013 (2011), USA [35] | None specified | None specified | Not evaluated | ||
Hannan 2012 (2012), USA [36] | None specified | None specified | Not evaluated | ||
Harris-Luna 2018 (2018), USA [37] | None specified | None specified | Not evaluated | ||
Process evaluation | The process evaluation used both quantitative phone-based surveys and qualitative in-depth interviews. | Informal assessment of user experience, acceptability, feedback from participants via text messages (during intervention) | In-depth semi-structured interviews with a sub-sample to explore user experience, perception and acceptance (during intervention) | •Behaviour modification likely if information is from a credible source such as from health professionals | |
Healthy Babies: Horodynski 2011 (2011), USA [39] | Process evaluation stated in study protocol paper, no evidence of one being conducted | Proposal to conduct: Feasibility, fidelity, and educational effectiveness of interventions. Mothers’ satisfaction with the Healthy Babies intervention; Recruitment; retention; | Proposal to conduct only - not published | Proposal to conduct interviews - not published | Not evaluated |
Process evaluation | A 3-phase process during planning and development | Baseline questionnaire survey to understand potential intervention approaches | Qualitative interviews with mothers midterm and at the end of the intervention | • Behaviour modification likely if information is from a credible source such as from health professionals • Delivery of interventions via text messages facilitated sharing of messages with family and friends • Lack of personalisation of contents in text messages | |
Process evaluation | Parents’ satisfaction with different aspects of the intervention will be assessed at 6mo via questionnaire | Questionnaire at 6 months to assess parents’ satisfaction with intervention | Semi-structured interviews with sub-sample of intervention and control group participants and facilitators to explore barriers and facilitators | • All participants reported the trial participation as a positive experience • They shared various experiences of not getting external help, support, or information about formula-feeding • Most notably, the most positive outcome of the trial participation for the mothers, probably not captured in the trial’s quantitative outcome measures but a central finding in this qualitative study, was the personal and non-judgmental support they received for formula-feeding | |
Process evaluation not specified. Acceptability measured | Assessment of App usage and App acceptability | Participant views | Qualitative follow-up interviews with parents | • Behaviour modification likely if information is from a credible source such as from health professionals • More likely to make changes if the content delivered aligned with their pre-existing beliefs • Level of engagement with programs fluctuated, based on their needs at a particular point in time and based on their child’s stage of development • Appealed to first-time mothers • Preference for a combination of delivery modes (eg., text messages, telephone calls, emails, push notifications, web, group sessions) | |
Smart Moms: Nezami 2018 (2014), USA [43] | None specified | None specified | Not evaluated | ||
Patel 2018 (2010), India [44] | Process evaluation not specified | Process evaluation not specified | Not measured | None specified | Not evaluated |
WIC: Pugh 2010 (2003), USA [45] | None specified | None specified | None specified | None specified | Not evaluated |
Tahir 2013 (2010), Malaysia [46] | None specified | None specified | None specified | None specified | Not evaluated |
Baby’s first bites: van der Veek 2019 (2019), Netherlands [47] | None specified | None specified | None specified | None specified | Not evaluated |
Mothers & Others: Wasser 2017 (2013), USA [48] | None specified | None specified | None specified | None specified | Not evaluated |
Healthy Beginnings: Wen 2007 (2007), Australia [2] | Process evaluation not specified | None specified | Questionnaires to participants only to evaluate infant feeding such as duration of breastfeeding, introduction of solids and healthy feeding practice | None specified | Not evaluated |
Process evaluation in protocol | Documentation of contact with families by intervention nurses; recruitment data barriers and enablers; study retention and intervention acceptability; interviews with participants to assess program satisfaction; identify emerging issues | Satisfaction questions administered at the 6-month and 12-month surveys | Interviews with participants to assess program satisfaction; identify emerging issues (during intervention) | • Consented to participate due to convenience of receiving interventions via telephone calls or text messages. • Behaviour modification likely if information is from a credible source such as from health professionals • Delivery of interventions via text messages facilitated sharing of messages with family and friends • Level of engagement with programs fluctuated, based on their needs at a particular point in time and based on their child’s stage of development • Preference for a combination of delivery modes (eg., text messages, telephone calls, emails, push notifications, web, group sessions) • Participation via telephone calls and by text messages was convenient • Appealed to first-time mothers • Considered themselves well supported through participation in program • Participants’ lack of time to participate due to return to work • Lack of personalisation of contents in text messages • High expectations placed on them as mothers | |
Linked trial for HB: Wen 2019 (2019), Australia [19] | Process evaluation of telephone contact with mothers (stated in study protocol, no evidence since study was ongoing) | Thematic analysis of participants’ responses (de-identified) will be evaluated retrospectively | Ongoing | Ongoing | Ongoing |