Process and effect evaluation of a referral aid for smoking cessation counselling in primary care: Findings of a randomized controlled trial

INTRODUCTION This study explored the use and effects of a smoking cessation referral in: 1) practice nurses (PNs), and 2) smokers. The use of evidence-based smoking cessation interventions (EBSCIs) can double the likelihood of a successful smoking cessation attempt. A referral aid was developed to aid Dutch PNs in primary care in deciding which smokers are the most suitable for EBSCI. METHODS Two different studies were conducted: 1) a randomized controlled trial with a process evaluation (n=82) and effect evaluation (n=285) among smoking patients recruited by PNs (n=73), and 2) a process evaluation among a subgroup of PNs (n=40) from January 2019 to September 2020. RESULTS Overall, the response in both groups was low. PNs found the referral aid materials clear and understandable. Smokers had similar but (slightly) less-positive opinions. The smokers in both groups did not differ in the amount of discussion and use of EBSCIs, nor on smoking abstinence. CONCLUSIONS Further research should assess how to better involve PNs and smokers when recruiting for an RCT and how to foster effective counselling. Additional research should also look deeper into barriers to referral of both PNs and smokers, and how to stimulate referral to EBSCIs best and help smokers to make a decision; for example by implementing a simplified strategy both within the primary care setting and outside, by involving other healthcare professionals or options outside healthcare such as the workplace and social domain. Trial registration The study was registered at the Netherlands Trial Register (NL7020, https://www.trialregister.nl/trial/7020).


INTRODUCTION
Smoking is responsible for 13% of Dutch morbidity, resulting in about 20000 deaths per year 1 and causing a burden of around €33 million in healthcare costs, decreased work productivity, and premature death 2 .Consequently, several actions have been undertaken to support smoking cessation at the policy level (e.g.public smoking restrictions and regular tax increases) 3 , the organizational level (e.g.national smoking restrictions in workplaces and smoking cessation interventions specially targeted at organizations) 4 , and at the individual level, for instance via mass media campaigns 5 or via healthcare such as the primary care settings (PCS) 6,7 , midwives 8 , nurses working on coronary wards 9 or other healthcare professionals

METHOD Design and intervention
The referral aid was named the 'StopWijzer', which can be translated as both stop-indicator and stopsmarter.The study consisted of a multi-site, twogroup, parallel-randomized controlled trial involving experimental and control conditions.The PNs in the control condition provided care as usual, by the seven steps from the Dutch treatment guideline for tobacco addiction and smoking cessation support 13 .The PNs in the experimental condition received an intervention manual to aid them in discussing smoking cessation with smokers and to help them select an EBSCI that fits the patient's needs and preferences (extension on step 5 of the Dutch Cessation Guidelines).An overview of the different parts of the study is given in Table 1.In the first part of the study, the recruitment of PNs and smokers, and the retention of the smokers, are described.In the second part, the use and appreciation of the referral aid materials from both groups are evaluated (process evaluation).In the third part, the effect on: 1) use of EBSCIs, 2) decisional conflict, 3) quality of life, and 4) abstinence and smoking behavior of smokers is evaluated (effect Measuring the effect on: 1) use of EBSCIs, 2) decisional conflict, 3) quality of life, and 4) abstinence and smoking behavior of smokers evaluation).A full description of the referral aid and the design of the RCT can be found elsewhere 23 .
The study proposal was approved by the medical ethics committee of the University Hospital Maastricht and Maastricht University (WMO, 2018-1038) and registered at the Netherlands Trial Register (NL7020, https://www.trialregister.nl/trial/7020).

Referral aid
Materials concerning the referral aid were delivered to the PN as a small (letterbox-sized) package sent via post.The content of the referral aid was based on a needs assessment comprising a literature review 6,13,24,25 , individual semi-structured interviews among GPs (n=5), PNs (n=20) and smokers (n=9), a Delphi study on the referral to EBSCIs 16 and the input of an advisory board consisting of experts representing various Dutch smoking cessation related organizations.The StopWijzer materials packages included the following items: 1.A manual (A4 size, approximately 20 pages), providing: a) an introduction and explanation of the aim of the referral aid; b) instructions on the use of the referral aid protocol, including a roadmap detailing the steps of the protocol and a flow-chart; c) an overview of reimbursement; d) an overview of the different readily available EBSCIs (face-toface counseling, eHealth, telephonic counseling, group counseling, NRT and pharmacotherapy) including discouraging remarks on the use of non-EBSCIs (acupuncture, hypnotherapy, laser therapy and the use of e-cigarettes as a means to quit); e) a short guideline for follow-up consultations; and f) concluding remarks and room for taking notes (Figure 1). 2. A separate handout (A5 size, printed on both sides) containing a visualization of the most important concepts of the manual (the same flow-chart as in the manual), and a summary of the health insurers' reimbursement policies.3.An overview of the different EBSCIs (option grid or decision matrix; A3 size, laminated placemat), explaining the target groups, strengths and weaknesses, effectiveness, and costs of the mentioned EBSCIs (Figure 2). 4. Supplementary materials for the promotion of the study include information flyers aimed at informing smokers about the study, business cards, posters (paper and digital), and a pen and notebook featuring the logo of the referral aids.
All materials were written in clear and comprehensible language in accordance with the applicable Dutch guidelines (language level B1) 26 and were also available on the referral aids' website (only accessible for the experimental condition).This website also included a frequently asked questions (FAQ) page tailored to both conditions.
PNs in the experimental condition were asked via the information letter to read the manual at the start of the study to inform themselves of the

Procedure of the study Part 1: Recruitment and retention of participants (smokers and PNs)
PCS were approached during January 2019 until May 2020 to recruit PNs to take part in the RCT (see Figure 3 for all time periods).PNs were recruited to: 1) recruit smokers, and 2) in the case of the experimental condition, refer smokers to EBSCIs in accordance with the method described in the referral aid.PNs were eligible if employed by at least one general practice in the Netherlands and providing smoking cessation counseling at least once a week.
A study invitation letter and a summary of the referral aids aim were sent to Dutch PNs using three approaches: 1) to three Dutch primary care associations (PCAs) in the south of the Netherlands collaborating with Maastricht University; 2) to individual PCS in the rest of the Netherlands via post and, when publicly known, via email (two This figure displays an option grid of all EBSCIs.On the top row, the different forms of EBSCIs are displayed (face-to-face counseling, eHealth, telephonic counseling, group counseling, NRT, and pharmacotherapy).The first row displays different categories in which the EBSCIs can be prepared, namely: target group, side effects, percentage of successful quit attempts, costs, benefits, and possible disadvantages.The three black rows in the middle read: 'What is it?','The use of multiple quit methods (such as combining personal guidance with nicotine replacement or medication) increases the chance of quitting' and 'Want to know more about costs, reimbursements, and the deductible?Use the handout about reimbursements or visit www.stopwijzer.nu' .
Tob. Prev.Cessation 2024;10(January):2 https://doi.org/10.18332/tpc/176148weeks after the initial recruitment message, a minimum of three attempts via telephone per PN were made to achieve a more active and personal form of recruitment); and 3) via national congresses and advertisements in trade magazines or websites of relevant organizations, e.g. the Dutch 'Quality register for smoking cessation (kwaliteitsregister stoppen-met-roken, www.kabiz.nl)'.
PNs expressing interest in participating were sent a more detailed information letter for the study and were asked to sign a study participation form.PNs were randomized in a 1:1 ratio on the practice level, in order of registration.As PNs from the experimental condition were provided with the referral aid and PNs from the control condition were only asked to provide care as usual (no additional intervention), blinding of the PNs was impossible.
Participating PNs were each requested to recruit 10 to 20 smokers [based on the sample size calculation of 292 patients with an effect size (odds ratio) of 0.30, a power of 0.80, and an alpha of 0.05] 23 .To stimulate active recruitment and prevent attrition, PNs were offered remuneration based on the number of recruited smokers (up to €100 for recruiting >15 smokers).To facilitate recruitment of smokers, regular contact by phone was maintained with PNs who did not register smokers, to remind them of participation and to provide them with tips from other PNs to recruit smokers who are not motivated to participate in the study.In addition, PNs received several other reminders, such as personal postcards and monthly newsletters, which were tailored by name and number of recruited smokers.The newsletter included personal success stories and recruitment tips from other participants, as well as recruitment tips based on literature.
At the end of the recruitment period (September 2020), all participating PNs were invited via email to take part in a process to evaluate the course of the RCT.The email provided a link to an online questionnaire and a summary of the referral aid and associated materials.The questionnaire took 15 minutes to complete, excluding the time PNs from the control condition needed to review the materials.PNs who did not respond within seven days were sent a maximum of two reminders.On completion, PNs received a €20 gift voucher as reimbursement.
The recruitment of smokers for the RCT occurred from May 2019 until May 2020.PNs were requested to inquire about the smoking habits of all smokers they spoke to during their consultations.Inclusion criteria for smokers were: use of tobacco products, aged ≥18 years, and able to read and understand the Dutch language.Those who only used e-cigarettes were not eligible.
Smokers who were eligible and willing to participate in the study were registered by the PN and received an information letter on their participation.Then, they received smoking cessation counseling with or without referral advice, depending on the condition to which the PN was assigned.Written informed consent was obtained from all participants at the start of the baseline questionnaire.Smokers were semi-blinded, as they were unaware of the procedure of any other group than the one they attended.Smokers were recruited to fill in two questionnaires: one at baseline and one at follow-up at 6 months.Smokers who filled in both questionnaires were rewarded with a gift voucher of €10.

Part 2: Process evaluation
To measure the use and appreciation of the materials by both PNs as smokers, as well as the course of discussing the different EBSCIs, a process evaluation was conducted during and alongside the RCT (i.e.only in the smokers and PN of the experimental group in the RCT; more details in Supplementary file Table 1 and the protocol publication 23 ).

Part 3: Effect evaluation
The goal of the effect evaluation was to determine the referral aid's effect on: 1) use of EBSCIs, 2) decisional conflict, 3) quality of life, and 4) smoking abstinence and smoking behavior (more details in Supplementary file Table 1 and the protocol publication 23 ).

Data analysis
The qualitative data were analyzed using input from the open questions belonging to the process evaluation, which were summarized in the text.For parts 1 and 2, differences in the reporting of use of the materials were analyzed using Person's chisquared tests on data of PNs from the subsample, and smokers who quit smoking after the intervention and those who did not.Appreciation of the materials were analyzed using independent sample t-tests to test for differences between the same groups of PNs and smokers.
For the RCT, descriptive analyses were conducted to describe the sample characteristics.Dropout analysis using chi-squared tests and t-tests were used to detect differences between smokers retained at the follow-up at 6 months and those who dropped out.Pearson's chi-squared tests were used to compare intervention effects on the discussion of EBSCIs according to PNs and the actual usage of EBSCIs by smokers.Differences between conditions on 24-hour point prevalence abstinence, 7-day point prevalence abstinence, and 6-month prolonged abstinence, were assessed using Pearson's chi-squared test on complete cases and negative scenarios (intention-to-treat principle) 27 .Initially, depending on the progress of the recruitment, a cost-effectiveness analysis (CEA) and a cost-utility analysis (CUA) were also planned to be conducted 23 .

Part 1: Recruitment and retention
Practice nurses A total of 1663 PCS were approached to take part in the RCT of which 73 took part (4.4%).First, the recruitment of practices via the three participating PCAs resulted in 19 PNs out of 420 PNs associated with the PCAs (4.5%).Second, 1243 PCS that were not part of these PCAs were individually contacted.This resulted in 54 PNs (4.3%) willing to participate in the RCT.Attempts to contact potential participating PNs were sometimes cut off by the practice operator or assistant (the reasons provided included a demanding workload, upcoming employee leave, and previous or current participation in other studies).PNs who were reached but did not want to participate explained that they did not have the time, were on special leave within the RCT-period, or had recently moved or would move practice.
Third, the referral aid was promoted at two national congresses and via advertisements in trade magazines issued by the participating universities or smoking cessation associations.This did not yield any PCS or PNs willing to participate.

Participating smokers
From May 2019 till the end of May 2020, the 73 participating PNs recruited 285 smokers to take part in the RCT.Although PNs were each asked to recruit Tob.Prev.Cessation 2024;10(January):2 https://doi.org/10.18332/tpc/176148 at least 10 smokers, recruitment rates varied widely between PNs.A total of 20 PNs did not recruit a single patient (n=11 in the experiment condition and n=9 in the control condition).Of the PNs that did recruit patients, PNs in the experimental condition (N=28) recruited an average of 6.12 smokers (SD=4.9) in comparison with 5.04 smokers (SD=4.8)by 25 PNs in the control condition.This difference in the number of patients per PN was not significant.
Of the total 285 participants registered by the PNs, 157 participants filled in the baseline questionnaire, of which 105 were included by PNs in the experimental condition and 52 participants by PNs in the control condition (see Figure 3 for more details and retention rates).The recruitment rate, as well as the retention rate at 6 months, did not significantly differ between the experimental and control conditions.

Part 2: Process evaluation Practice nurses
Recruitment among PNs for participation in the process evaluation yielded 40 PNs: 22 in the intervention condition and 18 in the control condition (Table 2).The process evaluation showed that PNs from the experimental condition used the placemat to describe the different available EBSCIs and discuss the details of their advantages, disadvantages, costs, and use.The digital poster, which they could display on a screen in their waiting room, was used the least.These PNs reported a relatively high appreciation of the materials, resulting in a score of 8.8 (SD=0.9).
All PNs indicated discussing possibilities for smoking cessation counseling in the GP setting, NRT, and pharmacotherapy.Counseling via an external smoking cessation coach was discussed the least among both PNs from the experimental

Participating smokers
Flyers and paper posters were seen or received by more than half of the smokers in the experimental group.Around a quarter of all smokers indicated that they saw the digital poster in the waiting room, discussed the placemat during the consultation with their PN or visited the website during the consultation.Furthermore, smokers appreciated the materials [score of 8.0 (SD=1.8),range: 1-10].Use and appreciation did not differ between smokers who ceased smoking after the intervention and those who continued smoking.For more details of the process evaluation see Table 3. Smokers in the experimental group reported more often than smokers in the control group that NRT, group counseling and eHealth were discussed.eHealth and group counseling were not mentioned in the control condition at all (Table 3).

Part 3: Effect evaluation
Table 4 summarizes baseline characteristics and follow-up at 6 months of smokers from both conditions.Smokers from both conditions did not differ on any of the measures at baseline or at followup at 6 months, including their use of EBSCIs to support their smoking cessation attempt.Dropout analysis did not find significant differences between smokers followed up and smokers lost to follow-up at 6 months.

Effect on abstinence and smoking behavior
As a large portion (48%) of data at the measurement at 6 months was missing, resulting in a disproportionate distribution of a low number of participants in both conditions, multiple imputations or multi-level analyses could not be performed on the data set 28 .We therefore report both complete cases and single imputation based on a negative scenario 27 (Table 5).The group of smokers who indicated to have not smoked a cigarette in the last 24 hours (24-hour point prevalence abstinence) was identical to the group of smokers who reported not having smoked a cigarette in the last 7 days (7-day point prevalence abstinence), and therefore not presented separately in Table 5.
We found no significant differences between the two conditions in either scenario for 7-day point prevalence abstinence and 6-month prolonged abstinence.

DISCUSSION Part 1: Recruitment and retention
Of the 1663 approached PNs, only a small percentage (4.4%)were willing to participate in the study.
Previous studies show such a low percentage is not uncommon for research within the PCS 6,29,30 .
As the PNs in the study recruited a small number of smokers, the recruitment period had to be extended, making the initial 12-month measurement unfeasible.It was, therefore deleted.As suggested by others 31 , we tried to stimulate early recruitment success through postcards with motivational messages, a newsletter (read by 40% of the participating PNs), and telephone calls.The inclusion of financial rewards did not seem to improve recruitment either.
Besides recruitment of smokers, retention rates are also important in RCTs.We had a retention rate of 44.9% (n=157) at baseline and 47.8% (n=82) at 6 months.These retention rates are comparable with other studies with little direct patient-researcher contact 6,31,32 , despite the use of multiple dropout prevention strategies used, such as sending several reminders for each follow-up questionnaire, promising respondents a €10 voucher for completing all follow-up questionnaires and using abbreviated follow-up questionnaires only assessing three questions regarding smoking behavior to nonresponders.Unfortunately, due to privacy reasons, we could not determine the reasons for drop-out.

Part 2: Process evaluation
Participating PNs reported a higher percentage of usage of all materials than patients, except for the

Part 3: Effect evaluation
We did not find different effects between the experimental and control condition on smoking cessation and actual usage of EBSCIs after referral.
Although smokers during the experimental condition were introduced to a wide variety of EBSCIs, their scores on the decisional conflict scale did not differ significantly from the smokers in the control condition.As most EBSCIs do not differ much on aspects such as risks or losses, this may explain the lack of conflict between both groups.Another study in a similar sample suggested that smokers may have already made their choice for an EBSCI before addressing smoking cessation with their PN, based on experiences from their environment, their own previous experiences, and the media 33 .

Strengths and limitations
This study had several strengths, including proactive outlining of effective smoking cessation methods to both PNs and smokers, addressing the low consensus on EBSCIs among HCPs, and focusing on the effectiveness of the referral aid, materials' appreciation, and recruitment process.Furthermore, about half (58%) of the smokers with low education were included, a group often difficult to reach 34 , and achieved a high cessation rate of over 50%.Other studies have faced similar recruitment barriers within or via the PCS 29,35 .
Our study has limitations.The limited study sample resulted in an inability to perform multilevel analyses or other statistical analysis while assuring a high statistical validity and possibly preventing type III errors (i.e.correctly rejecting the null hypothesis but for the wrong reasons, for example, when the intervention was not properly implemented).We therefore chose to consider a more descriptive approach to investigate our data, in contrast to the approach described in the protocol publication 23 .Another way to prevent a type III error from occurring, other than including a larger sample, is to monitor more strictly how the intervention is implemented by the HCP.This can be done through self-reporting by PNs or by observation by a trained researcher.However, valid self-reporting requires a lot of time and effort of the PN and might evoke socially desirable answers, producing a distorted picture.Observation by a trained researcher was not possible because of the COVID-19 pandemic and the associated distancing measures.To provide further insights, we recognize the importance of offering additional information on the frequency with which PNs discussed EBSCIs and the percentage of patients involved in these discussions.A comparative analysis of patient responses would enhance the understanding of the intervention's impact.
Second, our PNs participating may have been a select group who are more open to innovations or are more interested in smoking cessation-related healthcare (selection bias).A consequence might be that the results could be even less positive in a broader population.As PNs often report nonadherence to the Dutch Cessation Guidelines because of time or cost constraints 15 , another explanation for the low participation rate might be that PNs are discouraged by the burden of the additional research elements associated with RCTs.Reflecting on the reasons provided, it is imperative for future research to delve into these reasons comprehensively.Understanding the factors influencing PN participation is crucial for refining recruitment strategies.We acknowledge that out of the 1600 PNs approached, some may not have even read the invitation due to the high volume of research requests they receive.Exploring more Tob.Prev.Cessation 2024;10(January):2 https://doi.org/10.18332/tpc/176148personalized recruitment strategies is essential for future investigations.
Finally, the planned cost-effectiveness analysis (CEA) and cost-utility analysis (CUA) 23 were not executed because of the small sample size and lack of behavioral results (mainly, no differences in quitting behavior between both groups).

Recommendation for practice
In light of the issues described, we would like to propose three recommendations for practice: 1) recruitment within an RCT or other research study, 2) providing smoking cessation counseling and referral to EBSCIs within the PCS, and 3) providing smoking cessation counseling and referral to EBSCIs within the PCS outside the PCS.
First, our findings suggest that PNs find it difficult to recruit smokers for an RCT, possibly due to time constraints (heavy workload) or different priorities.This implies that alternative approaches should be considered, such as engaging specifically trained and compensated personnel to assist in patient recruitment in coordination with the PN and smokers.
Second, when looking at the situation within the PCS, time or cost constraints often play a large role in the adherence of PNs to the smoking cessation guidelines, including referral to EBSCIs 15 .Currently, referring smokers to cessation methods outside Dutch practices may imply that the PN/PCS will not receive the patient-related smoking cessation reimbursement from the patient's healthcare insurance, as these methods may not be covered by the patients' policies.Hence, exploring the feasibility of incorporating smoking cessation coaching outside the general practice into healthcare insurance could help alleviate the burden on the PNs, including the coaching of smokers to be included in the reimbursement system.
Third, it may be important to explore whether there are additional venues outside the PCS to talk about smoking cessation to divide forces, reach (dividing the responsibility) and persuade more smokers to quit (by spreading the message through multiple sources), for example, through other HCPs such as dentists, midwives or social workers, or through community health workers and the social domain.
Although research has found other HCPs also encounter barriers such as lack of time and training 10 , spreading the workload can help lower the total individual pressure.To achieve this, appropriate educational options, possibly a simplified version of the 5As or Dutch guidelines such as the ask-adviserefer (AAR) strategy, have already been tried out or proven effective in other settings 9 .Furthermore, HCPs should be able to claim reimbursement for these actions requiring additional adjustment to the current funding system of the Dutch Health care 36 .Other entryways for reaching smokers need further attention as well, such as the workplace 6 , the Internet or directly to known smoking households in the social domain 5 .

CONCLUSIONS
This study aimed to explore the use and effect of referral aid from the perspectives of PNs and smokers by investigating the course of recruitment and conducting a process and effect evaluation.Recruitment of both PNs and smokers resulted in low levels of participation.Overall, PNs found the materials clear and understandable.Smokers had similar but (slightly) less-positive opinions.However, the referral aid was marginally used, and the groups of smokers and smokers who quit did not marginally differ on discussion and use of EBSCIs, nor differed on abstinence.As the main finding concerned a low level of participation and use of the referral aid by PNs, further research should aim to assess how to better involve PNs and smokers when recruiting for an RCT and how to foster effective counselling.Additional research should also look deeper into barriers to referral of both PNs and smokers and how to best stimulate referral to EBSCIs and help smokers make a decision, for example, by implementing a simplified strategy such as the AAR, both within the PCS and outside the PCS, by involving other HCPs and options outside healthcare such as the workplace and the social domain.

Table 1 .
Overview of the different parts of the RCT Part and adherence rate of PNs at the outset of the RCT Smokers 285 Tracking the recruitment and adherence rate of smokers at recruitment, baseline and at follow-up at 6 months Part 2: Process evaluation PNs (subsample) 40 Evaluating the use and appreciation of the referral aid materials by the PNs Smokers 82 Evaluating the use and appreciation of the referral aid materials by the PNs Part 3: Effect evaluation Smokers (same sample) 82

Figure 3 .
Figure 3. Recruitment process during the research

Table 2 .
Process evaluation among practice nurses (use and appreciation of the materials and intervention effects on the discussion of EBSCIs) *1=strongly disagree; 5=strongly agree.Tob.Prev.Cessation 2024;10(January):2 https://doi.org/10.18332/tpc/176148therapy (n=9), and hypnosis (n=8).PNs indicated that although these options were discussed, this happened mostly at the request of the patient and without the endorsement of the PNs themselves.

Table 5 .
Effects on abstinence and smoking behavior per condition