Committed to Quality

PDF Download free Adobe Acrobat Reader.

WebFOCUS Streamlines the Curriculum at the University of North Texas Health Science Center

	SNAPSHOT Organization: The University of North Texas Health Science Center (http://www.hsc.unt.edu) trains osteopathic physicians, public health professionals, scientists, physician assistants, and other health professionals, injecting more than $244 million into the Texas economy each year. A 110-member physician group manages 190,000 patients in the Fort Worth area each year. The center also conducts leading-edge research through six Institutes for Discovery Challenge: To streamline the process of collecting and distributing course evaluation results to various constituencies Strategy: Web-based collection, analysis and distribution of information that can be entered and retrieved from virtually any location Results: Dramatic reduction in the time required to process and deliver evaluation results – from days to seconds Information Builders Solution: WebFOCUS, WebFOCUS ReportCaster, FOCUS, iWay

Attending medical school is an expensive proposition, causing medical students to plan their time very carefully. Thanks to Information Builders WebFOCUS Business Intelligence environment, the task of evaluating courses has become much easier at the University of North Texas Health Science Center (HSC). IT pros at HSC used WebFOCUS to create an Internet-based course assessment program that delivers customized reports tailored to individual needs.

"Many people depend on our department for information, including students, instructors, course directors, department chairs, deans – all the way up to the president of the institution," explains Jerry H. Alexander, Ph.D., associate professor of Medical Education and director of Academic Information Services at HSC. "By offering candid assessments and detailed feedback on the curriculum, students can quickly share insights to help us maintain and enhance curriculum quality."

The University of North Texas Health Science Center is dedicated to the advancement of all three disciplines of medical science: education, research and patient care. The 15-acre, $71 million complex is located in the Cultural District of Fort Worth, Texas. The institution has a combined faculty of more than 200, a staff of 900 and a cadre of 300 volunteer community physicians.

Alexander works in Academic Information Services in the Department of Medical Education, the department responsible for academic computing. Its mission is to support the HSC's commitment to excellence in education by providing a wide range of academic information support services to a broad constituency of the center's administrators, faculty, staff, and students. In practice, that takes the form of extensive testing services, external examinations, information services and statistical services. "We generate a lot of data daily," says Alexander.

Diagnosing the Requirements

The HSC has always given students the opportunity to evaluate instructors and courses. In the past, however, course evaluations were tallied and distributed using paper reports that had to be printed, sorted, packaged, labeled and mailed. "It was slow and cumbersome," admits Alexander. "We needed to streamline the data collection process so we could get evaluation results to our various constituencies in a more timely manner."

Academic Information Services considered a number of turnkey reporting solutions and several report development tools before settling on WebFOCUS. "We are a creative team that designs and develops much of our own software," continues Alexander. "We have lots of experience with Information Builders technology. When we learned about WebFOCUS, we were excited at the prospect of migrating and enhancing our existing FOCUS software and expertise. We recognize the leadership Information Builders has exhibited in the 1980s and 1990s and we trust that the company will remain state-of-the-art."

HSC purchased WebFOCUS, WebFOCUS ReportCaster, and iWay in August 1999. "Information Builders' offerings ranked above other business intelligence tools in several categories," Alexander recalls. "We liked the robust reporting capabilities, such as the ability to create custom reports and to dynamically color-code output variables based upon pre-determined values. Much of our output is graphical, so we also liked the embedded graphics capabilities so we don't have to export the data to another vendor's product. The ability to drill-down on values for purposes of elaboration was another attractive feature."

HSC enlisted help from Information Builders Consulting for the initial migration from DOS to Microsoft Windows NT. The organization also purchased a training package of advanced reporting techniques. "We had limited experience with HTML and we learned as we went along," says Alexander. "WebFOCUS was relatively straightforward to learn."

Building the System

The development team quickly gained proficiency with the WebFOCUS software. Using this new expertise, they set out to design and implement Quality, a system to collect and distribute course and instructor evaluation data in real-time via the Internet. There were two phases to the project: collection of the data from students and distribution of the data to the various constituencies.

The collection process was tackled first. According to Alexander, this was no small undertaking. Unlike traditional undergraduate education, where there is one instructor per course, most medical school courses have ten to twenty instructors. The collection process is further complicated in that medical students in their third and fourth years of study perform medical clerkships at hundreds of locations throughout the country. "The paperwork can be overwhelming," Alexander says.

Developers also designed Web forms for inputting course evaluation data. This way, instead of filling out a paper form that had to be manually tallied, students could evaluate courses and clerkships by responding to a series of online questions. They also developed a picture gallery of instructors to make it easy for students to identify all the instructors who taught a particular course.

"Quality was designed to bridge the gap between evaluation and curriculum planning through computerized course and instructor evaluations," says Alexander. "We analyze the data with FOCUS, using a five-point Likert-type scale to create a satisfaction index for every course and every instructor. Having a fourth generation language underlying WebFOCUS is very powerful."

Crunching the Data

Today, evaluation reports are online for all courses in all HSC schools, allowing students to enter and retrieve information from anywhere there is a computer with Internet access. Quality also lets students supply daily curriculum comments to course directors and anonymous e-mail to course faculty. Course comments are accumulated and categorized as relating to organization, presentation, instructor/student interaction, pace, workload, tests, support materials and services. These daily curriculum comments allow the course director to make adjustments while the course is in progress.

Following analysis, FOCUS databases are updated and reports are created to summarize evaluations for a course or clerkship as well as for each individual instructor. Composite reports are also created, which average instructor results for a given course. These reports serve as an important benchmark for instructors to compare individual evaluations. Report elements are color-coded, with statements rated most highly shown in green and those rated lowest shown in red. Additionally, a "satisfaction index" is calculated for each course or clerkship and each instructor.

Alexander says the new system saves hundreds of hours in staff time and thousands of sheets of paper each year. Clerks no longer have to collect and enter about 65,000 data points from evaluation forms. Additional savings stem from the instantaneous delivery capabilities of email and the Web, which makes the processes of hand-sorting reports, packaging them in envelopes, labeling the envelopes, and transporting them to the mail room obsolete.

"The time required to process and deliver results dropped from days to seconds," says Alexander. "With electronic reporting, what used to take 30 hours now takes just a few minutes. We formerly had to distribute course evaluations manually. Now WebFOCUS automatically creates and archives HTML reports, which are immediately accessible to authorized users. There are no longer any distribution chores."

WebFOCUS is deployed on a Microsoft Windows NT server. Numerical data is analyzed and stored in FOCUS while student registration data resides in an Oracle database on a UNIX platform, accessed via iWay. Long comments are stored in a PostgreSQL database on a Linux system and are retrieved using PERL scripts. WebFOCUS combines these various types of data dynamically each time reports are generated.

Looking Ahead

At present, the Quality system supports the evaluations of 741 students for about 200 required and elective courses within the HSC and several hundred evaluations of faculty, who teach in multiple courses. When fully developed, the Quality system will include evaluations for courses and instructors, clinical clerkships and preceptors, and post-graduate residents and faculty. It will also feature an extensive testing module for maintaining item statistics and grade rosters, along with a Student Progress area for tracking current and historical grades, student progress, and class rank. "We use conditional styling on the matrix," says Alexander. "It is color-coded so problem areas can be quickly spotted."

Alexander believes that the extensive business intelligence capabilities of WebFOCUS will become increasingly more important over time. "We plan to use WebFOCUS to compare the satisfaction index and do trend analysis across years," he adds. "For example, we can use it to learn how a particular course compares with others, how certain clinical training sites compare with others, and the relative strengths and weaknesses of evaluation elements within sites. We have been impressed with Information Builders' commitment to enhance the product and incorporate new Web developments as they become known."

David Baum is a freelance writer based in Santa Barbara, CA.