Sustainability planning in public health

1 INTRODUCTION

The successful implementation of a public health project, programme or intervention is merely the first step towards contribution of the individual project to better health and wellbeing for the target population in a specific public health domain (1)(2). Public health resources are limited, with decision-makers, providers, investors and other stakeholders focusing ever more frequently on the cost-effective allocation and distribution of time, human resources and funding (3). If we wish to reduce the burden of disease and improve the quality of life of the target population as far as is possible, and given the limited resources available, successful projects must be sustained over the long term (4). This means that the positive effects of the project continue even though some of the project activities have been completed. To achieve that essential goal, public health experts and teams would have to use a specific process, called the sustainability planning, which is in more detail defined below. The field of sustainability planning suffers from a lack of research and is frequently complex and full of challenges (4)(5)(6). Even projects whose implementation has been highly successful can come to an end when financial support runs dry, the human resources are no longer in place or the sustainability planning process has been terminated (7). Bodkin and Hakimi (7) point out that only between 40% and 60% of projects are sustained over the long term in the field of health promotion. This means the loss of the aforementioned valuable capacities (time, funding, human and technical resources) and decline in the positive effects on the health of the population, and the loss of the trust, support and cooperation of project staff, relevant stakeholders at different levels and in the local community. This can hinder future community mobilisation in other interventions, and lead to a loss of confidence in public health professionals and their host institutions (4)(7). Moreover, some researchers also wonder whether it is even ethical to develop and implement an effective public health project if one is aware at the outset that the support is insufficient to enable its continuation (5).

The challenges we encounter in sustainability planning reflect the lack of research in this field at one side and the characteristics and the nature in which public health operates at the other (4)(5)(8). In the field of sustainability there is a lack of research, which is the consequence of multiple challenges, such as: project implementation itself might be unsuccessful (in which case there is nothing to be sustained on a permanent basis) (4) (3); the quantitative monitoring of the success of implementation and of sustainability requires multisectoral cooperation of different stakeholders (4); while sustainability studies require long-term monitoring of the programme, even over several years, which is also usually not provided (4)(8). In practice, the lack of research in the field of sustainability planning means that different terms are applied to describe the concept of sustainability. There is no uniform definition of sustainability, and while there are several sustainability tools and frameworks, they are rarely validated (4)(5)(7).

Schell et al. (9), who upgraded the definition produced by Scheirer and Dearing (10), define the sustainability of a programme or intervention as the ability to maintain a project or its components, activities and positive effects on the health of the target population over the long term, leading to the achievement of the desired health outcomes or the desired objectives of the project over time. The project sustainability is more accurately defined as a process and not merely as a final outcome. By looking at the project sustainability merely in linear terms as the final outcome achieved (e.g. the sustained positive effects on the health of the target population and the sustained project activities), we neglect its other dimension: that sustainability is also a process of constant adjustment, improvement, learning, development and change in line with the needs of the system, environment or target population, which we call ‘sustainability planning’ (3)(7).

Sustainability planning is therefore active planning designed to ensure project sustainability over the long term, and a process carried out at the implementation phase and continuously throughout the project’s life cycle and beyond. During the sustainability planning process, attention must be paid to numerous factors within and outside the programme itself that can affect its sustainability (7). Sustainability frameworks or tools can help us to assess the project’s long-term sustainability potential (5). An assessment of the potential for sustainability is followed by the formulation of the sustainability plan, a document that serves as support and guidance for staff when considering why the priority project areas have been selected, what steps have been taken and what we wished to achieve (11).

The aim of the systematic literature review was to establish the extent to which the field of sustainability planning has already been studied, what dictates whether a project will be successfully sustained and how a sustainability plan should be written. To achieve this aim, we set ourselves the goal of reviewing the available academic literature in the field, examining which definitions of project sustainability are applied, identifying the factors that affect sustainability and exploring the available sustainability planning tools and frameworks.

2 METHODS

The document search for the systematic literature review in the field of sustainability planning in public health was conducted in the PubMed bibliographical database (12) and on the website of the Centers for Disease Control and Prevention (CDC) (13). Before that decision was taken, a preliminary search in the ScienceDirect bibliographic database was also conducted, where we mainly found articles on the individual practical experience of different projects with sustainability planning. After initial trials we decided to focus on theoretical articles on sustainability planning instead, especially in the field of or linked to public health, so we conducted a search in the PubMed bibliographical database. This greatly narrowed down the number of relevant hits, so we decided to conduct another preliminary search on the WHO (World Health Organization), ECDC (European Centre for Disease Prevention and Control) and on the CDC (Centers for Disease Control and Prevention) websites. The hits on the WHO and ECDC websites focused mainly on sustainability in terms of the environment and communicable diseases rather than sustainability planning in the field of public health or health promotion, while we wanted to focus on the latter, so a search was conducted on the CDC website which was more promising with such search results. The publications review was conducted in accordance with PRISMA (Transparent Reporting of Systematic Reviews and Meta-Analyses) guidelines (14). The review considers articles in English published between 2011 and 2021.

Before consensus was reached among the authors on the selected search string, a preliminary search of bibliographic databases was performed using different keywords in different syntaxes. First, a preliminary search in PubMed was performed. Keywords "program sustainability" yielded the most hits. If the keywords "project sustainability" were used, the articles that were found related to the practical experience of individual projects in sustainability planning. The key words "sustainability plan/planning/management" yielded hits related to environmental sustainability, whereas the authors wanted to focus on theoretical aspects of sustainability planning in the field of public health or health promotion. A preliminary search in ScienceDirect yielded similar results, with the hits referring to the experience of individual projects or to the environmental sustainability. By testing different search strings, we wanted to increase the number of relevant hits. In the end, a consensus was reached among the authors to conduct a document search in the PubMed bibliographical database twice using key words in two different syntaxes: (program sustainability[Title/Abstract]) AND (health promotion) and ((program sustainability[Title/Abstract]) AND ((public health) OR (health promotion)). The search term ‘program sustainability’ was only sought in the title or the abstract, while the other key terms were sought in all fields of the bibliographical database. An advanced search was conducted on the CDC website using the key words ‘sustainability planning’ (‘This exact word or phrase’). The title, abstract and full paper review was done by one author.

The review considered original and review academic articles and other documents (guides and guidelines) focusing on the theory of sustainability planning in the field of public health or health promotion. Here we were interested in the definition of sustainability, the factors that affected the successful maintenance of public health programmes and interventions, and the theory underpinning the formulation of a sustainability plan. Documents that addressed the environmental aspect of sustainability and those that detailed experiences with sustainability planning in specific projects or programmes were excluded from the research. The overall conceptualisation of the paper, inclusion and exclusion criteria and discussion with conclusions were discussed between both authors.

The systematic review also included documents that we searched for using the snowball method, i.e. in the list of sources that were mentioned at the end of the selected articles from PubMed and that met the criteria for inclusion and exclusion. The results of the publications included in the systematic review are processed in tabular form with a presentation of the key findings.

As stated in the introduction, three aims are identified in the systematic review: 1) to establish the extent to which the field of sustainability planning in public health has already been studied, 2) what dictates whether a project will be successfully sustained i.e. what are the factors that influence project sustainability), and 3) how a sustainability plan should be written. The answer to the first research question and partially the answer to the second research question can be found in the results section. In order to equip the reader with practical information on how to write a sustainability plan and to empower them to improve the sustainability of the public health projects they are involved in, the authors decided to fully answer the second and third research questions in the discussion section rather than in the results section in tabular form. Thus, the findings from the articles included in the systematic review could be analysed and synthesized more in-depth.

3 RESULTS

The first search of the PubMed bibliographical database yielded 77 hits with the first search string and 271 hits with the second search string. By limiting the number of documents by including only English-language articles, published between 2011 and 2021, the number of hits in the PubMed bibliographical database was reduced to 48 publications with the first search string and 165 publications with the second search string. After reviewing the articles by title, we excluded 168 publications that addressed the environmental aspect of sustainability and those that detailed experiences with sustainability planning in specific projects or programmes. The number of relevant hits in PubMed was therefore reduced to 45 publications. This was followed by a review of the abstracts, which led to the further exclusion of 31 publications that addressed experiences with sustainability planning in specific projects or programmes and that did not focus on the theory of sustainability planning. In line with the inclusion and exclusion criteria, 14 articles suitable for the whole document review remained in the last phase, with a further six articles subsequently being excluded. This produced a total of eight sources suitable for inclusion in the final analysis.

A total of 116 hits were obtained during the first publication search on the CDC website. After reviewing the documents by title, we excluded 104 publications that did not relate to the topic at hand or that were in an unsuitable form. The number of relevant hits on the CDC website was therefore reduced to 12 publications. This was followed by a whole document analysis, which led to the exclusion of further seven documents, leaving five sources suitable for inclusion in the final analysis. The final analysis therefore included a total of 13 documents from PubMed and the CDC website. This was followed by the purging of duplicates, which led to the further exclusion of seven publications. Using the snowball method, and in line with the inclusion and exclusion criteria, the sources in selected articles from PubMed and the CDC website were further examined, with nine sources deemed to be suitable for inclusion in the systematic literature review. This gave a final total of 15 documents to be included in the systematic review. Figure 1 is a diagram, showing the precise process by which documents were selected for the systematic literature review. The results of the analysis of the publications included in the systematic review are shown in Table 1.

Figure 1 Diagram of the precise process by which documents were selected for the systematic literature review

Eleven documents were published in the USA (4)(9)(10)(11)(15)(16)(17)(18)(19)(20)(21), three in Australia (6)(22)(23) and one in Canada (7). Of the 15 documents included in the systematic review, nine were academic review articles (4)(7)(9)(10)(11)(15)(17)(18)(23), two were academic research articles (6)(20) and four were guides (16)(19)(21)(22). All the publications included addressed the field of public health or health promotion.

The most cited definition of sustainability was that advanced by Scheirer and Dearing (4)(7)(10)(17), which was subsequently upgraded by Schell et al. (9)(11). Eight documents offered their own definition of sustainability (6)(15)(16)(18)(19)(21)(22)(23), while one scientific research article did not provide a definition of sustainability (20).

The most common distribution of sustainability factors into subgroups was that of Scheirer and Dearing (10), which was subsequently upgraded by Schell et al. (4)(9)(11)(20). The other documents included in the systematic review defined the factors affecting sustainability in a very similar way, although they did place them into different subgroups (6)(7)(15)(16)(18)(19)(21)(22)(23). One of the articles did not focus on the factors affecting sustainability, but on the types of public health intervention that help us assess the impact of individual factors more precisely (17).

The Program Sustainability Assessment Tool (PSAT) is the most commonly recommended sustainability framework. Four articles refer to it (4)(7)(11)(20), with the work of the authors of three of the articles providing the basis for its creation (9)(10)(17). The seven other documents recommended other tools (4)(15)(16)(18)(19)(21)(22). One document recommended another tool alongside the PSAT (4). Two articles contained no recommendations regarding the use of sustainability frameworks (6)(23).

Additionally, based on the findings from the articles included in the systematic review individual steps in the process of sustainable planning from a practical point of view were defined (preparation (11)(16); assessment of the current potential of a project for long-term sustainability using the Program Sustainability Assessment Tool (PSAT) (4)(7)(9)(10)(11)(19)(20); deciding which elements of the project to retain (11)(19)(22); determining the priority domains or factors affecting the sustainability of the project) (11)(16). Based on the findings from the articles included in the systematic review, also a template for writing a sustainability plan was compiled (11)(16)(19)(22). Due to the complexity and length limitation, these findings are not presented in the tabular form, but are analysed and synthesized in detail in the discussion section, where they have also been upgraded with additional references.

Table 1: Table with results of individual studies. CDC = Centers for Disease Control and Prevention; BMI = Body mass index; PSAT = Program Sustainability Assessment Tool; OPA = Office of Population Affairs; USA = United States of America. *As the study is of a scientific research nature, we have presented the results differently (it is not appropriate to give the same column headings used for the other publications).

Tabela 1

4 DISCUSSION

Sustainability planning is a dynamic process and one that should be launched early on during the project or program planning phase and continued after successful implementation and for the entire duration of the project or program, which applies also for the sustainability planning in public health and health promotion. Researchers recommend that before sustainability planning is commenced, a precise definition is to be produced of what sustainability means to us and to our project and what precisely we aim to sustain over the long term (e.g. project activities, collaboration between stakeholders, positive effects on the health of the population). In doing so we can make use of one of the commonly cited definitions of project sustainability from the literature, or adapt it ourselves and then apply it. We then opt for one of the sustainability planning frameworks or tools as the project gets under way. If possible, the framework or tool should be validated and should take account of the key factors that affect the sustainability of the project in question in terms of the wider context in which it operates. This is followed by the writing of the sustainability plan and its practical implementation, and then by the regular reassessment of the potential for the long-term maintenance of the project (3)(7)(11).

4.1 DEFINITION OF PROJECT SUSTAINABILITY

A clear understanding of what the concept of sustainability means for us and for our project or program is the basis for the successful commencement of sustainability planning. Basic precondition is that we have an effective and successfully implemented evidence-based project that is worth maintaining (10). We find ten different definitions of project sustainability in the documents included in the systematic literature review. The most cited definitions of sustainability are those advanced by Scheirer and Dearing (4)(7)(10)(17) and Schell et al. (9)(11). Many researchers recommend that a definition of sustainability to be chosen is the one that is most relevant to our circumstances. This can then be further adapted and designed in accordance with the specific context of the project or program in question (7)(16)(21)(23). This is what Whelan et al. did in their own research, designing a very narrow definition of sustainability specific to the obesity prevention field (23).

Using the academic review paper of Moore et al. (28), while taking into account the possible limitations of the review, we for our purposes employed five aspects of the definition of sustainability that appear in the documents included in the systematic literature review: 1) time (e.g. long-term maintenance of a project after the end of financial support), 2) continuation of project activities, elements or components, 3) continuation of the positive effects of the project (e.g. continuation of the desired health outcomes, achievement of the objectives and project results at the individual or systemic level), 4) behavioural changes at the individual level among staff or in the community (e.g. change to an individual’s working practices), and 5) project or program evolution and adaptation (e.g. adjustment and development of project activities or staff practices). Practically all definitions of sustainability referred to or formulated in the documents included in our systematic review include the aspect of continuation of project activities, elements or components (4)(6)(7)(9)(10)(11)(15)(16)(17)(18)(19)(21)(22), while slightly fewer contain the aspects of time (4)(6)(7)(9)(10)(11)(16)(17)(18)(22) and continuation of the positive effects of the project (4)(6)(7)(9)(10)(11)(16)(17)(18)(21). The aspects of behavioural changes at the individual level (6)(15)(16)(22) and project evolution and adaptation appear more rarely (16)(18). Moore et al. (28) produced very similar results in their analysis, and used the results of their research to formulate a comprehensive and useful definition of project sustainability that sheds light on all its aspects: project sustainability means the long-term continuation of project activities, elements or components after the phase of successful implementation or the end of financial support, whereby the acquired behavioural changes at the individual level among staff or in the community are maintained, continuing the positive effects of the project at the individual or systemic level, which can include the evolution and adaptation of programme activities or staff practices (28).

4.2 FACTORS THAT INFLUENCE PROJECT SUSTAINABILITY

The ability to identify and understand the many factors that affect the long-term maintenance of a project or program is important to the sustainability planning process. In the discussion we produced a synthesis of the recognised factors from the articles included in the systematic literature review and, following Scheirer and Dearing (10), placed them into three main groups. In this process we made use of the framework division advanced by Bodkin and Hakimi (7). The synthesis presented below concerning factors that influence project sustainability does not include the article by Luke et al. (20), which discusses the reliability of the PSAT and is included in synthesis concerning sustainability planning in Chapter 4.3.

It is clear from the analysis performed that the long-term maintenance of a project depends on several factors and not only on the financial support secured as it is usually suspected. According to the findings of our systematic review, a public health project that has the potential to be sustained over the long term successfully maintains partnerships between many stakeholders from different sectors (4)(6)(7)(9)(10)(11)(15)(16)(18)(19)(21)(22)(23), can draw on adequate staff capacities (4)(7)(10)(11)(15)(16)(18)(19)(21)(22)(23) and financial support (4)(7)(9)(11)(15)(16)(18)(19)(21)(22)(23), operates within a favourable wider socio-economic and political climate (4)(6)(7)(9)(10)(11)(15)(16)(19)(21), has political support (7)(9)(11)(15)(16)(18)(19)(21), is adaptable (4)(7)(9)(10)(11)(18)(21)(22)(23), is regularly evaluated (7)(9)(11)(15)(16)(18)(22)(23), is proved to be effective (4)(7)(9)(10)(18)(21)(22), has been carefully planned (4)(7)(9)(16)(18)(21)(22), accords with the main mission and policy of the host organisation (4)(6)(7)(10)(18)(21)(22), communicates successfully with stakeholders (7)(9)(11)(16)(18)(23), is capable of strengthening its capacities when required (4)(7)(16)(18)(22)(23), has a project champion (4)(7)(10)(16)(18)(21)(23), and addresses needs in the community and promotes social and health equity (10)(16)(22). Whelan et al. (23) and Wiltsey Stirman et al. (18), who coded, from the available academic literature, the factors that affect the sustainability of projects, also found that the most commonly cited factors in the literature related to the areas of stakeholder integration, staff and financial capacities.

4.2.1 Factors pertaining to the internal context of a public health project

Programme/project adaptation is the ability of a project to be modified and improved in order to maintain effectiveness. A project may be adapted to changes in the host institution, the latest scientific approaches, and conditions in the local and wider environment or the target population. This process enables the key elements of the project to be maintained and increases the likelihood that it will be sustained over the long term (4)(7)(9)(10)(11)(18)(21)(22)(23).

Programme/project evaluation is the process of collecting, monitoring, and analysing data connected to project activities and outcomes. It requires certain specific organisational capacities. Using evaluation methods, we assess whether a project is effective in its current form and whether it is worth sustaining over the long term. If evaluation capacities are inadequate, this reduces the ability to report on the project results achieved. This makes it more difficult to convince staff, key individuals, stakeholders, and decision-makers of the value of the project. The findings of the evaluation process form the basis for the adjustment of project activities to meet the needs of the target population as far as possible (7)(9)(11)(15)(16)(18)(22)(23).

Strategic planning is the process of the prior determination of the desired directions, objectives and strategies of the project, and prior sustainability planning (including the funds required for this). Good strategic planning, together with the insertion of the project within the host organisation’s existing organisational structure and work tasks, makes a vital contribution to ensuring project sustainability, allowing for the institutionalisation of action (4)(7)(9)(16)(18)(21)(22).

A project that accords with the host organisation’s main mission and policy, as well as the needs, priorities and opinions of the community, has a better chance of receiving the necessary support from the host organisation and of continuing over the long term (4)(6)(7)(10)(18)(21)(22).

If the effectiveness of the project is clearly promoted and communicated to the target population and providers, with details provided of the ways in which it makes a positive contribution to improving public health, this increases their understanding of the project (and consequently increases the chances of the project continuing over the long term). The project therefore helps to change health-related behaviours and attitudes towards health within the target population (4)(7)(9)(10)(18)(21)(22).

Communications refer to the strategic sharing of project outcomes, results and activities with stakeholders, decision-makers and the public. The regular sharing of experiences, negotiations regarding stakeholder requests and demands, the prompt resolution of problems and the search for the key factors that will enable project sustainability (a search conducted with decision-makers and partner organisations in the implementation period) all increase the likelihood of the project being sustained over the long term (7)(9)(11)(16)(18)(23).

4.2.2 Organisational capacity

Organisational capacity comprises the internal support and resources we require for the effective management of a project and its activities (4)(9)(11).

Staff capacities refers to sufficient number of employees, stability of staff, adequate staff qualifications, strong, capable and charismatic project leadership, the appointment and involvement of internal project managers, the appointment of a leadership group (not just one project manager), the support of the host organisation’s leadership and management, and a belief on the part of key staff that the project is beneficial (4)(7)(10)(15)(16)(18)(19)(21)(22). Good project leadership is trusted, knows how to allocate human, financial and other resources, encourages new developments and approaches, and fosters cooperation between staff and stakeholders (23). The characteristics of key staff (their knowledge, skills, values, goals, motivation, attitude and commitment to the project, the prevailing climate within the host organisation) are important factors in the long-term maintenance of a project (4)(11)(15)(18)(23). Staff problems, such as the lack of suitably qualified staff, difficulties in retaining good members of staff, rapid staff turnover, overworked staff with too many responsibilities (e.g. they work in the field as well as plan the project), can hinder the ability to sustain a project over the long term (7)(15)(23). Aarons et al. (15) point out that staff capacities must be carefully planned if a project is to be sustained over the long term (15). If possible, existing staff capacities should be used (23).

A programme champion is an influential individual who works as a committed advocate of the project and its requirements (particularly financial). Most often this is an influential expert in a management position within the host organisation, although it could also be a key partner or stakeholder. The programme champion knows how to bring management or key employees on board and ensure that the organisation’s internal policymakers support the project and provide him or her with the resources necessary for ensuring that the activities can continue (4)(7)(10)(16)(18)(21)(23). The loss of a programme champion can have a very negative effect on the sustainability of the project (23).

If we wish to plan and maintain a project over the long term, we must know how to mobilise existing sources of funding and, where necessary, secure new ones. Only in this way can funding stability be secured. The search for different sources of funding is a major challenge in the public sector and one that can distract staff from other tasks, thereby having a negative effect on the project, or overburden staff with project implementation so that they neglect sustainability planning, which includes considerations regarding funding (4)(7)(9)(11)(15)(16)(18)(19)(21)(22)(23). Access to other possible sources of funding from the wider environment, such as grants, is another factor than can help to sustain a project over the long term (4)(10).

Capacity building means the creation of conditions for project success at the individual, project, host organisation and systemic level. To this we can add the education and training of (new) staff, the strengthening of the host organisation’s structure and of commitment to the project, and the swift identification and resolution of problems. All these factors increase the likelihood of a project being sustained over the long term (4)(7)(16)(18)(22)(23).

4.2.3 Factors pertaining to the wider external context of a project

Partnerships are the connections between a project and numerous stakeholders from different sectors (e.g. from the healthcare sector, politics, academia, the community and staff) that aid the pooling of resources for the achievement of a common objective. The support and successful integration of different stakeholders and organisations have an important impact on the long-term project sustainability, and provide additional human, financial or technical resources, skills and knowledge. As far as integration with organisational and community stakeholders is concerned, project sustainability relies on a feeling of inclusion, cooperation, joint decision-making and the opportunity for stakeholders to make their own contributions from the project planning stage onwards (4)(6)(7)(9)(10)(11)(15)(16)(18)(19)(21)(22)(23).

Direct political support, i.e. where individual political figures work as advocates of and support the project, can aid the formulation of policies that support the funding, operation and public acceptance of the project. This increases the likelihood of the project being sustained over the long term. If the project loses political support or even becomes a political target, this can lead to loss of wider public support, thereby reducing the possibility of its long-term continuation (7)(9)(10)(11)(15)(16)(19)(21). The opposite also applies: the more the project is inserted into the existing political sphere, the greater the likelihood that it will receive political support and be sustained over the long term (18).

The wider socio-economic and political climate is a factor that we cannot, in most cases, control. Nevertheless, it has a considerable impact on the project sustainability, which means that we should consider it when planning the project and in the course of its subsequent life cycle. The replacement of governments, short budget periods, domestic political pressures, long-term policy formulation, poverty, high levels of unemployment and criminality, poor economic conditions, a deterioration on the workings of government institutions and a weak healthcare system (particularly characteristic of developing countries) can all jeopardise the long-term sustainability of projects. The local community will be more inclined to take a positive view of a project that is capable of being adjusted to traditional and cultural values. This in turn will make it easier to sustain the project over the long term (4)(6)(7)(9)(10)(11)(15)(16)(19)(21).

Shelton et al. (4) point out that while there is a clear dynamic link between the different factors that affect project sustainability, there has been no thorough research into how these relationships play out or an examination of which factors are more important than others, which factors can be absent and which factors are critical (4). Scheirer (17) states that the role played by specific factors depends mainly on the type of public health intervention or project involved. This means that strong, responsive leadership capable of bringing together and representing all stakeholders and of fostering a feeling of involvement on the part of all participating organisations is key to the long-term maintenance of coalitions or partnerships with stakeholders. Where partners value the work and contribution of all relevant stakeholders, this maintains motivation and therefore participation in the coalition (17), in which in the case of public health projects public health drivers for action have to be clearly defined. Shelton et al. (4) also add the importance of good, timely sustainability planning and the ability of a coalition to adapt to political, staff and financial changes.

4.3 SUSTAINABILITY PLANNING

4.3.1 Preparation

Before starting the sustainability planning activities, the team responsible for writing must be defined. It might be beneficial if part of its members are holding management positions or are heading project activities or represent important stakeholders. They will, as they start drafting the plan and with the help of the PSAT, assess the potential for the long-term project sustainability, and become aware that sustainability planning is a protracted but necessary process. The team holds discussions, takes the necessary decisions relating to the future of the project and ends by implementing the sustainability plan. A team coordinator is appointed to ensure that the sustainability planning process has a suitable and realistic timetable and that the right collaborators and stakeholders are included, plan team meetings, oversee communication within and outside the team, and supervise implementation of the actions subsequently defined in the plan. Devil is in the details and we are presenting some of those which could at the end influence the final sustainability efficiency: when drawing up the timetable, the coordinator takes into account the information and technical resources available, as well as the roles taken by the team members in line with their skills and interests; the coordinator assesses when the team should meet in person and when a video conference suffices (11)(16).

The sustainability planning team first defines the sustainability of the project in question, which means that they review the short- and long-term objectives of the project, providing a precise definition of the form that the sustained project will take in three to five years and what precisely the project is to achieve. A clear long-term vision makes it easier to attract more stakeholders into the project. It also makes it easier to determine, in subsequent steps, what is needed to be done to ensure the long-term sustainability of the project (11)(16).

4.3.2 Assessment of the current potential of a project for long-term sustainability using the Program Sustainability Assessment Tool (PSAT)

Sustainability frameworks or tools serve as guidelines that help to assess and improve a project’s long-term sustainability potential. They are based on a close examination of those groups of factors that affect the project sustainability (7)(11). In addition to the PSAT, the documents included in our systematic literature review mention seven other sustainability frameworks or tools (4)(15)(16)(18)(19)(21)(22). According to our findings, the PSAT is so far the only validated sustainability framework or tool to contain a reliability assessment (7)(20). As it is referred to in most of the documents included in the review (4)(7)(11)(20), we present it in detail below. It is accurate, user-friendly and free, and is commonly used in a variety of public health contexts (7)(11). The PSAT is based on the groups of factors first mentioned by Scheirer and Dearing (10). Schell et al. (9) later upgraded them and made a more precise classification of them into eight groups or domains: support of the internal and external environments, funding stability, partnerships, organisational capacity, project evaluation, programme adaptation, communications and strategic planning (9). The PSAT defines project sustainability as the ability to maintain programming and its benefits over time. It therefore guides users through the steps of identifying the benefits and challenges of specific projects in terms of sustainability. It can be used for programmes, projects and interventions at different levels (local, regional, national, EU) in the fields of public health, social sciences and education, and is also suitable for assessing international programmes (11)(29).

Definition of the activities to sustain and the choice of assessors

Before completing the PSAT, all assessors should know the project and its host organisation well and know which precise activities are to be sustained and assessed over the long term. This is particularly important if the project involves external partners and if the aim is also to preserve and assess a coalition of stakeholders. The project assessors should be key individuals who know the project well, are committed to it and have a wide perspective regarding the benefits and challenges of the project (e.g. experts in the field, volunteers, project management, host organisation management, stakeholders, community representatives). This step is crucial because it compels stakeholders to consider which projects activities are critical and which are not and to uncover any differences in viewpoints (11)(19)(29).

Completion of the PSAT

When the assessors have been appointed, they are invited to complete the PSAT. The PSAT has five questions in each of the eight domains, i.e. a total of 40 questions. For each of these questions the extent to which a specific element is present in the programme, using a scale of 1 (to little or no extent) to 7 (to a very great extent), is accessed. The PSAT can be completed on paper or online (29). Completion takes between 10 and 15 minutes and the assessment can be made by an individual or group of assessors (11)(29). At the end of the assessment, the website automatically compiles the results for the individual assessor and for the group as a whole and creates an automated final report for the individual or group (30)(31).

As the author, Washington University permits adaptation of the tool (e.g. paraphrasing, the addition or removal of questions) as long as the tool is not used for commercial purposes in its amended form and reference is made to the original author. They do warn that the reliability of the PSAT can be compromised if it is amended (29).

One positive advantage of the PSAT is that it is easy to use, although this might also be regarded as one of its limitations. In case the eight PSAT domains do not cover all the important factors that affect the sustainability of individual assessed project, a wide-ranging and open discussion within each of the eight domains should be added to the assessment process (4)(11). Shelton et al. (4) point out that it would be a good idea to validate the PSAT for use in a variety of contexts as well.

Factors which have the greatest impact on the sustainability of the project and the domains or factors with low results regarding the project’s long-term sustainability potential should be given greater attention (11).

4.3.3 Deciding which elements of the project to retain

The assessment team compares the project’s realistic current long-term sustainability potential (assessed using the PSAT) with the project’s long-term objectives as defined during preparation for sustainability planning in section 4.3.2 (reflecting the ideal or desired future status of the project). Based on that, elements of the project could be preserved, adjusted or removed, taking care to ensure that the elements selected enable the desired objectives or the project’s long-term positive effects on public health to be achieved, and that the key stakeholders are ready for further cooperation (11)(19)(22).

4.3.4 Determining the priority domains or factors affecting the sustainability of the project

Key questions for determining the priority domains or factors of sustainability are: (1) where we are at present; (2) what we would like to achieve and (3) what we have to do to achieve the sustainable goals. These considerations lead towards identifying the obstacles, i.e. the factors that reduce the sustainability potential of the project, or deciding which PSAT domains and indicators will be addressed as a matter of priority so as to increase the project’s long-term sustainability potential. Usually, the domains with the lower PSAT results are given priority, along with those that the team deem to be more important within the context of the project. Low hanging fruits are then considered first, regarding the resources available (11)(16).

For each of the priority domains clear objective(s) are set to increase the project’s sustainability potential, and to determine the precise steps to achieve those objective(s). The responsible person or stakeholders, the measurable milestones and the necessary resources and time periods are defined at each step (32).

4.3.5 Writing a sustainability plan

A sustainability plan is a document that brings together all the information gathered at previous steps. The content is provided by all members of the sustainability planning team, with the document itself being compiled by one or two people. Updates to the document are reviewed by everyone as they arise, with the final document being approved by all members of the group. It is therefore a dynamic document that defines the future path of the project or program and contains details of why and how the decisions regarding the project’s or program’s long-term sustainability have been taken. It should be written as concisely as possible, with all the documents gathered during the sustainability planning process being enclosed at the end. A sustainability plan is a strong communication document. CDC researchers recommend the Formulation of two documents: a detailed sustainability plan with precisely delineated strategies for internal use, and a second shorter overview document for external use that is shared with external partners, stakeholders and decision-makers for knowledge transfer and communication purposes. The shorter overview document indicates the ways in which stakeholders can support actions and what precisely is expected from the stakeholders if the sustainability objectives are to be met, among others. Foreseeing any possible conflicts arising between internal or external forces (e.g. stakeholders, the system) and listing possible solutions is advisable (11)(16)(19).

The literature contains a variety of freely available sustainability plan templates. These can be used also in public health and in health promotion to compile a very detailed or merely indicative sustainability plan. A template for the table of contents of a sustainability plan that combines the recommendations of the CDC (16) and the PSAT website (33) is presented below. Any third relevant format could be used, too (29):

a. Summary:

Short presentation of the project or program, its current status and a presentation of the host organisation, a description of the long-term vision of the project or program (description of long-term objectives from section 4.3.1), a list of current funding sources with the expiry dates of the support, a summary of the benefits and challenges of our project in terms of sustainability (we have identified them with the help of the PSAT, section 4.3.2), a list of project elements that we have decided to retain (section 4.3.3), a list of associates or partners that took part in the sustainability planning process (the sustainability planning team and the people who completed the PSAT), and the time frame in which the activities referred to in the next point will be implemented;

b. Precise analysis of the situation

• The long-term objective(s) for each of the priority domains that we defined in section 4.3.4 (the change that we would like to see in this domain and that will raise the sustainability potential of the domain). The objectives should be SMART: specific (what, who), measurable (how many changes we expect – we quantify the expected change or determine a reference point from which the change can be measured), attainable (the objective must be achievable in the given time frame with the resources available), relevant (adequately addresses the scale of the problem and capable of being realised in the given time frame with the help of the actions described in the next point) and time bound (the deadline by which the objective will be realised or when successful implementation of the objective can be measured);

• The precise actions through which the long-term objectives will be achieved, consideration should be given to the current political and organisational conditions. The responsible person and the envisaged time frame for each action is determined;

• A list of the resources and stakeholders that we require for realisation of the actions (e.g. time, money, space, information, technical support, materials, specific employees, stakeholders or partners). Consideration should be given to the unique skills and resources that individual stakeholders can contribute and the ways in which they can support the sustainability of the project over the long term;

• The time frames and tangible measurable milestones for monitoring progress in the completion of the actions (11)(16)(19)(32).

4.3.6 Measures: implementation and evaluation

Preparation of a sustainability plan is followed by its implementation and the commencement of a dynamic process by which the outlined actions are constantly closely examined, the project or program activities are amended and adjusted, solutions are sought to the challenges encountered on the way, progress reports are produced, and close cooperation is fostered with staff and stakeholders. Researchers recommend that the team reassess the project’s or program’s long-term sustainability potential every year, compare the results and produce a report on the progress made that year and on the challenges faced. The sustainability plan can therefore be adjusted to take account of the remaining or new weak domains (11)(22).

4.5 LIMITATIONS AND POSITIVE ASPECTS OF THE RESEARCH

In this systematic literature review we have restricted ourselves to a relatively narrow interval of time (2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021) and to documents in English, which has narrowed the scope of the review to a certain extent. Systematic reviews must take account of the subjectivity of the researcher, who reviews and assesses the studies against the inclusion and exclusion criteria, according to the aim of the review, as well as the variability of the number of hits that a specific search string yields (as a result of the constant addition of new articles to online bibliographical databases). Despite certain limitations, this literature review is one of the few on this topic which are addressing public health and health promotion more specifically.

CONCLUSION

If we wish to define what sustainability means for a specific project, programme or intervention, we should try to take as comprehensive a view as possible and cover the aspects of time, finance, the continuation of programme activities, the positive effects on the target population, staff, the system and society, and the modification or development of project activities. Sustainability planning is a dynamic process that must be planned and monitored carefully, as the long-term project sustainability depends on many factors that can quickly become obstacles. Project sustainability is not a final endpoint. On the contrary, through constant monitoring the project can constantly be improved, the long-term objectives achieved and a contribution made to improving the health and wellbeing of the population.

Conflicts of interest: The authors declare that they are not subject to any conflicts of interest.

Funding: The authors funded the research from the annual action plan of NIJZ.

Reference

Reference

1. Vitale R, Blaine T, Zofkie E, Moreland-Russell S, Combs T, Brownson RC, et al. Developing an evidence-based program sustainability training curriculum: a group randomized, multi-phase approach. Implement Sci 2018; 13(1): 126.←
2. Whelan J, Love P, Pettman T, Doyle J, Booth S, Smith E, et al. Cochrane update: Predicting sustainability of intervention effects in public health evidence: identifying key elements to provide guidance. J Public Health (Oxf) 2014; 36(2): 347–51.←
3. Lennox L, Maher L, Reed J. Navigating the sustainability landscape: a systematic review of sustainability approaches in healthcare. Implement Sci 2018; 13(1): 27.←
4. Shelton RC, Cooper BR, Stirman SW. The Sustainability of Evidence-Based Interventions and Practices in Public Health and Health Care. Annu Rev Public Health 2018; 39: 55–76.←
5. Walugembe DR, Sibbald S, Le Ber MJ, Kothari A. Sustainability of public health interventions: where are the gaps? Health Res Policy Syst 2019; 17(1): 8.←
6. Harris N, Sandor M. Defining sustainable practice in community-based health promotion: a Delphi study of practitioner perspectives. Health Promot J Austr 2013; 24(1): 53–60.←
7. Bodkin A, Hakimi S. Sustainable by design: a systematic review of factors for health promotion program sustainability. BMC Public Health 2020; 20(1): 964.←
8. Wolfenden L, Chai LK, Jones J, McFadyen T, Hodder R, Kingsland M, et al. What happens once a program has been implemented? A call for research investigating strategies to enhance public health program sustainability. Aust N Z J Public Health 2019;3(1): 3–4.←
9. Schell SF, Luke DA, Schooley MW, Elliott MB, Herbers SH, Mueller NB, et al. Public health program capacity for sustainability: a new framework. Implement Sci 2013; 8: 15.←
10. Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. Am J Public Health 2011; 101(11): 2059–67.←
11. Calhoun A, Mainor A, Moreland-Russell S, Maier RC, Brossart L, Luke DA. Using the Program Sustainability Assessment Tool to assess and plan for sustainability. Prev Chronic Dis 2014; 11: 130185.←
12. US National Library of Medicine. PubMed [website]. Accessed 1 July 2021 at: https://www.ncbi.nlm.nih.gov/pubmed/.←
13. US Department of Health Human Services. Centers for Disease Control and Prevention [website]. Accessed 1 July 2021 at: https://www.cdc.gov.←
14. Page MJ, Moher D, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ 2021; 372: n160.←
15. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health 2011; 38(1): 4–23.←
16. Centers for Disease Control and Prevention. A Sustainability Planning Guide for Healthy Communities [e-book]. United States of America: National Center for Chronic Disease Prevention and Health Promotion, Division of Community Health, 2011. Accessed 1 July 2021 at: https://www.cdc.gov/nccdphp/dch/programs/healthycommunitiesprogram/pdf/sustainability_guide.pdf.←
17. Scheirer MA. Linking sustainability research to intervention types. Am J Public Health 2013; 103(4): e73–80.←
18. Wiltsey Stirman S, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci 2012; 7: 17.←
19. Buck K. The Path to Program Sustainability [e-book]. United States of America: Nonprofit Impact, 2013. Accessed 1 July 2021 at: https://silo.tips/download/the-path-to-program-sustainability.←
20. Luke DA, Calhoun A, Robichaux CB, Elliott MB, Moreland-Russell S. The Program Sustainability Assessment Tool: a new instrument for public health programs. Prev Chronic Dis 2014; 11: 130184.←
21. US Department of Health and Human Services, Office of Adolescent Health. Building Sustainable Programs: The Resource Guide [e-book]. United States of America: US Department of Health and Human Services, 2014. Accessed 1 July 2021 at: https://safesupportivelearning.ed.gov/sites/default/files/sustainability-resource-guide.pdf.←
22. Hill K, Vrantsidis F, Clemson L, Lovarini M, Russell M. Community falls prevention program sustainability guidelines and workbook [e-book]. Australia: National Health and Medical Research Council, Victorian Government Department of Health, 2011. Accessed 1 July 2021 at: https://www.monash.edu/__data/assets/pdf_file/0007/218581/nhmrc_guidelines_and_workbook.pdf.←
23. Whelan J, Love P, Millar L, Allender S, Bell C. Sustaining obesity prevention in communities: a systematic narrative synthesis review. Obes Rev 2018; 19(6): 839-51.←
24. Mancini J, Marek L. Sustaining Community-Based Programs for Families: Conceptualization and Measurement. Family Relations 2004; 53(4): 339–47. Accessed 1 July 2021 at: http://www.jstor.org/stable/3700173.←
25. US Department of Health and Human Services, Office of Adolescent Health. OPA framework for program sustainability [e-book]. United States of America: US Department of Health and Human Services, 2017. Accessed 1 July 2021 at: https://opa.hhs.gov/sites/default/files/2021-02/framework-for-program-sustainability.pdf.←
26. Johnson K, Collins D, Wandersman A. Sustaining innovations in community prevention systems: a data-informed sustainability strategy. J Community Psychol 2013; 41(3): 322–40.←
27. Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Educ Res 1998; 13(1): 87–108.←
28. Moore JE, Mascarenhas A, Bain J, Straus SE. Developing a comprehensive definition of sustainability. Implement Sci 2017; 12(1): 110.←
29. Washington University in St Louis, Brown School, Center for Public Health Systems Science (CPHSS). Program Sustainability Assessment Tool (PSAT) [website]. Accessed 1 July 2021 at: https://sustaintool.org/psat.←
30. Washington University in St Louis, Brown School, Center for Public Health Systems Science (CPHSS). Individual Sustainability Report – example program [website]. Accessed 1 July 2021 at: https://sustaintool.org/wp-content/uploads/2018/10/Example-Indiv-Report.pdf.←
31. Washington University in St Louis, Brown School, Center for Public Health Systems Science (CPHSS). Group Sustainability Report – example [website]. Accessed 1 July 2021 at: https://sustaintool.org/wp-content/uploads/2018/10/Example-Group-Report.pdf.←
32. Washington University in St Louis, Brown School, Center for Public Health Systems Science (CPHSS). Action Planning Template 1 [website]. Accessed 1 July 2021 at: https://sustaintool.org/wp-content/uploads/2017/01/SustPlanTable.doc.←
33. Washington University in St Louis, Brown School, Center for Public Health Systems Science (CPHSS). Program Sustainability Assessment Tool (PSAT) – Develop your plan [website]. Accessed 1 July 2021 at: https://sustaintool.org/psat/plan-for-sustainability.←