In the Evaluation section:
Evaluation
What Are Effective Ways Of Evaluating My eGovernment Website?
This depends on the goals you have set for it, and on how these can be measured.
The goals you have set may be of a quantitative nature (e.g. dealing with 80% or more of overall enquiries, generating a specific amount of income from tax or services, or reducing the time taken to answer enquiries by half). But goals may also be qualitative (e.g. establishing a high-profile image and reputation for an institution, or providing a higher level of service to citizens compared with traditional means). In either case, you should choose adequate instruments and parameters for your evaluation.
The following are examples of some of the tools at your disposal.
- Monitoring Website Traffic
- Monitoring Transactions
- Monitoring Visibility and Awareness
- Monitoring User Activity and Satisfaction
- Cost and Cost-Effectiveness
Monitoring Website Traffic
You can use specific software to monitor traffic on your website, i.e. keep a count of the actions initiated by users on your server within specific periods of time. In other words, you can extract usage statistics from the activity of servers. These will help you answer some basic quantitative questions, such as "how many users a day?", or "what is the most visited page / downloaded document / form filled?".
It should nevertheless be noted that unfortunately, due to the technical aspects of how the web works, it still isn't possible to obtain very precise statistics (e.g. users may have to make different attempts at loading a page or submitting a form before they are successful, or may save contents on their own PCs (hidden from your monitoring software) rather than consult web pages regularly at your site, or some of the visits may actually be made by "spider" software rather than real users (see: How Do I Maintain Awareness Of My eGovernment Website?).
In this respect, it may be safer to carry out overall comparisons over periods of time or between specific activities, rather than focus on absolute counts: e.g. it would be safer to state that "page xyz was visited roughly 20% more this month compared to last month" rather than saying it was visited "by x number of individuals"; the same goes for comparisons between sections or pages (i.e. you can monitor if a page or section is more used/downloaded than another, in relative terms). It should also be noted that you can make comparisons between overall usage statistics for different sites (provided the same data collection and analysis method is used on all of them).
Monitoring Transactions
As mentioned, monitoring traffic using web statistics does not provide very precise sets of data for your evaluation. In some cases, it may be more helpful to monitor the number of actual transactions (e.g. enquiries) that take place via a website. This is because transactions can be better defined, as actions that lead to specific and measurable outcomes. For example, if you have a web form that enables enquiries from users to be sent to a specific e-mail address, you could then count incoming messages, which would equate to successful enquiries generated by the website. This is particularly useful if your goals are in some way connected to the number of transactions handled via your website (e.g. if you set out to answer x number of enquiries each month).
It is likely you will want to connect the quantity of transactions also with other data, such as their growth over time, or the time taken to handle them, or the proportion of online enquiries compared to the ones logged via traditional methods. In this case, you may need to design or acquire specific information systems that accept and process all the relevant information and provide you with meaningful results. The scope of such operations goes beyond the aims of this text, but it is worth noting that the more complex the goals you set out to achieve, the more expensive and time-consuming it will be to measure and evaluate achievements, so you should always make sure that you budget appropriately for this kind of activity when you plan the project.
If possible, you should also try to monitor transactions that may have been initiated by a visit to your site, but then taken forward via other channels (e.g. people who visit your agency may be doing so after reading information on the site). This can be done by asking the individuals who interact with your organisation where they heard about the agency or about a new service, for instance (as it may have been via your website). This will help you estimate if and how the site is "creating business" for your agency, by facilitating contact with users in general, and inviting them to interact with your organisation, though not necessarily online.
Monitoring Visibility and Awareness
This is partly connected to the promotional activity you undertake for your site (see: How Do I Maintain Awareness Of My eGovernment Website?).
Part of the task of delivering a web project is also ensuring that as many potential users as possible are aware of its existence, and that the site is easy to find - as this is likely to contribute importantly to the achievement of the goals you have set.
Running a few searches in the most popular search engines, using different combinations of words related to what your agency deals with (but also simply inputting the name of your agency, for a start), can already give you an idea of how your site fares in terms of search engine ranking, and therefore visibility. Some search engines (e.g. Google) also allow you to have a rough count of the number of incoming links to your site, which is another important factor in the visibility and reachability of your site: but as with usage statistics, it is safer to take these figures with a pinch of salt, and to perhaps limit your analysis to relative counts (e.g. comparing to other sites, or between moments in time). Usage statistics will also provide you with information on what sites users are coming from when they access your site ( referrer logs ), i.e. it is possible to establish which incoming links are more effective and useful for you.
Another way to evaluate awareness in potential users is of course to ask them: for example you can run surveys with non-users (e.g. people who handle their relation with your agency via other communication channels) and ask them whether they are aware of the possibility of communicating with the agency through the web, whether they know the web address of your site, and so on. The same can be done with current users (generally easier and more cost-effective than with non-users), for example asking questions that help you understand the level of user awareness of different sections of the site, or of the different information and services you offer online.
You should also keep monitoring whether your site is listed on (and linked to from) other important sites in your area of activity, and solicit the inclusion of links to the site when new sites or information databases are made available in relation to your field of action (which implies generally monitoring the "web presence" of the sector in which you operate).
Monitoring User Activity and Satisfaction
This relates more to the patterns of usage of your site, and to the quality of the experience users enjoy when dealing with your agency online.
As mentioned, usage patterns can be estimated in quantitative terms by analysing server statistics. But these statistics will not tell you much about the preferences and expectations of the people who use your site: in this case it may help to run surveys with current users (e.g. by e-mail) asking them for example to rank the types of activity they undertake when visiting your website in terms of importance, or preference, or the sections they use the most, or any features they'd like to see implemented in the future. This will tell you whether your site is responding to user expectations, which services are more popular, and whether it needs to be improved or new goals need to be set. You could also analyse search logs , if possible, i.e. keep track of the keywords input by users in the search box of your site (most site search software usually provides you with this option): this will help you understand what it is your users are looking for the most (and because they are using the search feature to find it, it may mean that they haven't been able to find by navigating the site, which should push you to reconsider the navigation system you have implemented).
You should also try to monitor user satisfaction : as much as it is important that your site is used by many, it is also crucial that users are satisfied with the level of service you provide online, as this is likely to lead them to continue using the site, and therefore guarantee its sustainability. Again, user surveys can help: for example, you could ask users to express their level of satisfaction with the services they use, and provide them with ways to submit suggestions for the improvement of the level of service. It also helps to keep feedback channels permanently open (rather than simply run surveys, which only provide limited windows of opportunity for users to express their opinion): online feedback forms, clear indications as to how to make an enquiry about the site (as opposed to about specific services), and generally providing visible and user-friendly feedback mechanisms, are good and relatively inexpensive ways to extract information about the level of satisfaction of users of your site.
Cost and Cost-Effectiveness
It is likely you will have been assigned a specific budget for your project, and that you are supposed to achieve specific goals keeping expenses within such budget.
It may therefore be important that you monitor your expenses regularly and transparently, listing what is spent for what feature or function of the site, if possible.
This will help you calculate the cost-effectiveness of your site and of its different features. If, for instance, your site provides mechanisms to answer enquiries online (e.g. online forms), you can divide the number of enquiries by the cost of maintaining and attending to these mechanisms (both in terms of real money and staff time taken), and compare this with the cost of dealing with enquiries logged using other communication channels. If enquiries logged online "cost less" per unit than traditional enquiries, your site can be said to be a cost-effective way to address enquiries in general, which would in part justify the financial effort and investment needed to run it. If, on the contrary, online enquiries cause higher costs per unit (unlikely, but possible), you may have to review expenditure on the service, or consider ways to make it more cost-effective (or to abandon specific services).
If your site is designed to generate income of any kind (e.g. tax revenues, or licence fees), an analysis of the return on investment may also be applicable, whereby you offset the expense needed to offer a service with the income it generates (including any savings generated specifically by the fact that the service is offered online, as outlined above). This will help you calculate how much income is generated per unit of investment (e.g. $1.20 for every $1 invested), and therefore establish whether the site is contributing revenue rather than requiring expenditure, e.g. compared to other communication channels.
In general, monitoring cost-effectiveness will help you justify the existence of the project, and more importantly, is likely to provide you with solid grounds to solicit new funding in order to continue it or expand it.