• As described in the Project Management Section an "integrated project team development model" (see Diagram A) will be used to manage the project. An integrated project management team will coordinate four self-directed development teams, responsible for infrastructure, software multimedia, instructional design and virtual community development. These unique teams must achieve a high level of interdependence for the success of the project. This has direct implications on the evaluation design.
• There is a critical need for arms-length evaluation that will ensure the alignment with the overall project objectives and CANARIE guidelines.
• Formative evaluation techniques will ensure valuable feedback during the project and summative techniques will contribute to the overall project evaluation.
• There is a critical need to ensure a systemic coordination between and within the individual teams throughout the LearnCanada project.

The proposed evaluation methodology reflects the interdependence of the four development teams and the critical role that each one has in attaining the overall project objectives. Critical to the success of this project is an evaluation methodology that uses both formative and summative techniques. Formative evaluation techniques will be used to assure a timely and useful feedback within each team's objectives as well as to assure the coordination between the teams in meeting the overall project objectives. Summative evaluation techniques will be used periodically during the project, as well as its conclusion. The outcomes from the summative evaluation will be used to demonstrate how the project is meeting its objectives.

The evaluation will assess the following main outcomes of the project:

• The functionality of the broadband enabled learning environment
  • accessibility
  • response time
  • frequency and ease of use
  • meets user requirements
• The effectiveness of adult learning within the virtual learning environment
  • migration to innovative learning behaviour and culture
  • sharing of best practices
  • collaborative development of innovative pedagogy
• The effectiveness of middleware and multimedia objects and tools to support learning
• The need to sustain and expand a virtual community of national and international educational practitioners.

At the onset of the project benchmark metrics, (both qualitative and quantitative) will be established for each development team and the project as a whole. The established benchmark metrics will be used to track and monitor both the individual team and overall project objectives and evaluate the overall progress and success of the project. The management and evaluation teams will ensure the alignment (across teams) of these metrics with the CANARIE Learning program objectives.

• Benchmark the metrics using qualitative and quantitative research techniques. This will be done to establish the extreme points and internal scales for each metric continuum.
• Regular and periodic tracking and monitoring of the teams' and project's progress against the established metrics. This will include techniques such as interviews, questionnaires, self-evaluation, surveys, focus groups, facilitated meetings and naturalistic observation. The project participants will be the primary sample for data collection. On occasion, in order to get different viewpoints, the sample will extend to adult learners outside of the project.
• Ongoing and regular dissemination of results to all stakeholders.
• Final evaluation at project end to determine the degree of the project success against its overall objectives.

Given the innovative nature, complexity and objectives of the LearnCanada project, an evaluation methodology that serves the purpose of both assuring its on-going progress as well as the fulfillment of the objectives is required. In order to accommodate the innovative and unique nature of the project, the proposed methodology is grounded in the project, yet flexible to adapt to unforseen environmental changes. The complexity inherent in the LearnCanada project will be supported by the evaluation as casual relationships within and between the team activities will be identified in order to optimize resources.

The proposed evaluation methodology will ascertain the successes of the project based on quantifiable and qualitative evidence. The use of formative and summative evaluation techniques, regular and periodic evaluations will provide a framework for the ascertainment of the project's overall success. Appropriate and timely feedback is at the backbone of the success of this project.

The milestones for the evaluation component of the project will reflect the milestones of both the LearnCanada project and the individual teams. The evaluation team will ensure that the formative evaluation required within each team is applied effectively and efficiently. The evaluation team will assist teams in implementing and interpreting team evaluation results. In addition, the evaluation team will facilitate the between team evaluation process i.e. peer review of individual team progress.

The appropriate progress and results of the LearnCanada project will be published electronically for all stakeholders to access. In addition, participation in both national and international conferences will be recommended. Other forms of publications will also be considered.

Marion Barfurth Ph.D. has over fifteen years of experience in the area of innovative learning and teaching with leading-edge technology. Her research has focused on collaborative learning in a socio-constructivist theoretical framework. Her work has been recognized for its contribution to the understanding of the collaborative learning process. Andrew Katz has over ten years experience as a professional facilitator and consultant in the private and public sectors. Areas of expertise include evaluation criteria, product development and risk analysis. Andrew's client base includes leading telecommunication suppliers, national associations, as well as high-tech start-up companies.

The evaluation of the economic and social benefits will be evaluated through research in how the established results of the project can reduce costs and enhance professional development for educators and adult learners in general. Evaluation of the potential inherent benefits of broadband enabled learning environments to address the barriers for the development of Canada's human resources requirements for the knowledge economy, has both significant economic and social benefits for all Canadians.