1. Definition 

As a consequence of the standardization of platforms, storage media, communication protocols and distribution channels we define Electronic Publications in Medicine (EPMs) as follows: ·  

• The subject of EPMs is theoretical and practical Health-related generic knowledge. 
• The principle target users are professionals and students in the Health sector, as well as other groups interested in medicine, e.g. interested patients. 
• The goals of EPMs are to promote and advocate the acquisition, consolidation, dissemination and updating of practical and theoretical knowledge. 
• The term "Electronic Publications in Medicine" implies public accessibility. This includes commercial and public domain products, and is independent from the diffusion channels (WWW, CD-ROM). 

• software tools that do not transmit specific Health-related contents (text processors, presentation software, expert shells) 
• components of hospital information systems for patient-related data management, 
• statistic packages for analysis of medical or administrative data, 
• electronic or semi-electronic media that make use of analogous technologies, such as video tapes, audio tapes or slides. 

• Special input and output devices and their influence on the program control, since their availability cannot be taken for granted (Hardware). 
• Environments for distributed, co-operative learning. 
• Tools that allow the user to edit or manipulate the content of the publication. 

 2. Quality Criteria 

• domain competence, 
• competence in software engineering, 
• media competence, 
• design competence, 
• didactic competence. 

2.1. Contents 

2.1.1. Authorship 

• Contents are presented correctly and comprehensively. 
• The content is adapted to the specified target group . 
• Personal opinions are explicitly marked. 
• Authors, editors, release dates and version numbers are documented. 
• Quotations and reference sources are marked as such, attributed and documented. 
• Medical contents are linked to existing coding schemes and professional nomenclatures. 
• The contents are kept current through periodical updates. Knowledge-based systems are periodically updated by domain experts. 
• Commercial publications are either internally or externally reviewed and evaluated as in normal professional, scientific practice. 

2.1.2. Formal Requirements 

• Commercial publications are listed in catalogues and can be retrieved by their ISBN number. 
• Copyright information is available. 
• License agreements contain explicit information and permission with regard to multi-user operation, loan and rental or purchase. 

2.1.3. Target Group Reference 

• Target users, prerequisite skills and learning objectives are adequately specified. 
• If any computer knowledge is necessary, the kind of knowledge and its amount is clearly specified. 
• Scope and profundity of the contents are perceptible. 
• Where standardized curricula exist, it is referenced in educational software. 

2.2. Technical Aspects

2.2.1. Plattform

• The application is developed for the computer systems available among target users. Ideally, the more common systems are supported, to ensure as widespread use of the application as possible. 
• The system requirements and software constraints are clearly identified. 
• Applications only use fixed graphics settings (i. e. fixed resolution or color depth) if unavoidable. 
• Simultaneous usage on multi-user operating systems is accounted for. 

2.2.2 Hardware Constraints

• Hardware better than the minimum required (monitor, graphics adapter) does not impair the quality of presentation. 
• Hardware poorer than expected causes a warning to be displayed. 

2.2.3. Software Installation 

• The application can be started directly from the data medium without a setup routine.
• The application runs without modifying system areas of the operating system. 
• There is no need of system reboot or manual modification of the configuration to start the application the first time. 
• Where an installation routine cannot be avoided, all system modifications are clearly documented and an uninstall routine is available. 
• The software can be installed on a file server without the need of separate client installations. 
• The software  runs as a real client/server application with clients from different platforms (this applies mainly to databases). 

2.2.4. Performance / Flexibility / Runtime Characteristics / Interfaces

• The application is stable, robust, reliable and performs well. 
• In the case of applications designed for quick information loading delay is minimized. 
• Where considerable response delays cannot be avoided,  warnings are displayed. 
• Saving of user-specific settings, data and extensions (e. g. annotations) is supported by distinct and protected user profiles. 
• An auto-save function saves user-specific data recorded during runtime, in short intervals. 
• Interfaces with complex systems, such as hospital information systems or text retrieval systems are well defined and sufficiently documented. 

2.2.5. Special Criteria for Internet-based Electronic Publications

• There is a sensible compromise between ease of operation, user-friendly design and runtime performance. Realistic data transfer rates are taken into account. 
• Large pictures can be previewed ("thumbnails"). 
• Security issues are taken seriously when plug-ins are requested. Plug-ins are not used where there exists a good alternative. 
• Internet-based publications that do not communicate with processes running on the server have the option of an en-bloc download of the entire package for off-line use. 
• Intelligent combinations of on-line and off-line elements aimed at a reduction of communication costs are supported, as much as possible, when allowed by the content. 
• Where "gateways" to other Internet publications are intended the user is informed how to return to the original document.  A "go back " or "return" control is always available. 
• Those pages that belong to a defined Internet-based publication contrast by their appearance with other pages in order to minimize the risk of getting lost. 
• New browser windows are opened only if there is a real need and does not occur in an imperceptible way. It is best to maintain a distinct style difference between the main and all secondary windows. 
• Due to the volatility of the content of Internet-based publications and the relationships between internally and externally referenced WWW documents, the date of the latest update is contained on every page and modifications are documented. The consistency of external links is warranted by the author. 

2.3. Coding of Information and Modalities of Presentation

2.3.1. Text

2.3.1.1. Meta Information 

• The total size of the publication is transparent. 
• For each text unit (chapter) the total number of pages is indicated. 
• A hierarchical order of the elements in the document is supported by a logical numbering system. 
• If there exists more ordering principles, these are clearly displayed and can be easily accessed. One of them is marked as default. 
• Meta-information, such as authors, version number, summary can be viewed at any time without the need of leaving the current page. 

2.3.1.2. Formal Aspects 

• The contents are expressed concisely and compactly. 
• Texts are orthographically, grammatically correct and stylistically consistent, and punctuation is correct. 
• The elements of composed documents can be selectively addressed (e. g. for printing, saving and mailing). 

2.3.1.3. Layout 

• Layout, font choice and formatting show consistency. 
• Larger amounts of text use easily readable fonts, whereas small, highlighted text units may exhibit more a creative, individual typography. 
• Tutorial systems do not appear with screens full of text. Scrolling is avoided, as far as possible, and the rule "one topic - one window" is followed whenever possible. 

2.3.1.4. Acronyms, special terms 

• The use of acronyms is limited to those commonly understood in the subject or specialty area. 
• Where the training or retrieval of foreign terms is given special emphasis, an acoustic support of the pronunciation is available, or at least a phonetic spelling using the international phonetic alphabet (IPA) is given. 
• Rarely used terms are explained by a glossary or a lexicon. A glossary can be extended by the user. 

2.3.1.5. Hypertext 

• A hypertext based publication has powerful orientation tools that can be used to intuitively and intelligently navigate though applications 
• The "hyperspace" of the publication can be graphically visualized. Direct access to the information about the orientation tools is provided. 
• Hyperlinks are used carefully and parsimoniously. The semantics of hyperlinks is explained (typed links). 
• Where sequential reading can not be assumed, implicit textual references to previous discourse objects are avoided or made explicit by hyperlinks (as in frequent use of pronouns, or phrases like: "see above" etc.). 
• Advanced organizers (index pages with many hyperlinks) contrast stylistically with text pages (few or no hyperlinks). 
• Presentations make distinctly clear whether they offer loosely connected elements (menus, indexes) or whether they must be received as a whole, self-contained document. 
• The layout of "visited links" contrasts (normally by color) with that of other, not yet visited links. 
• The individual navigation paths are recorded in a "history" that can be viewed and stepped forwards or backwards. 
• Configurable bookmarks and notepad functions are provided. 

2.3.1.6. Retrieval 

• Users can search the complete content of EPMs with efficient retrieval tools. These tools take the user's behavior and requirements into account and, as an option, adapt to the user's interests during run time. 
• Retrieval tools support both index based and free text searches. They can be extended with Boolean operators and wildcards. 
• Retrieval results are sorted by relevance. 
• The usage of retrieval tools is explained in an comprehensive on-line help with examples. 
• Retrieval methods are able to process synonyms. 
• Retrieval methods take into account that medical terms do not always conform to orthographic norms. 

2.3.2. Graphics, Animation, Photographs, Video and Audio Sequences 

2.3.2.1. Graphics and Photographs 

• The choice of picture/graphic resolution and color depth is constrained by content on one hand and by the hardware and software commonly available among target users on the other. In spite of these compromises, pictures must be able to relay meaningful information. 
• Zooming pictures offer more visual information than a simple magnification of the single pixels. 
• Pictures are labeled with unique, context-independent remarks. 

2.3.2.2. Animated Pictures and Video Sequences 

• Animated sequences and video clips are mainly employed to enhance information acceptance and where they relay information better than with still pictures or text. 
• Animated sequences and video clips can be used to produce humorous or dramatic effects in the context of a thoroughly well planned user-interface design. They do not produce undesirable delays or interruption of didactic sequences. 
• Movie-like opening and closing sequences, when used, can be skipped over or switched off. 
• It is best to employ video sequences only where the user's computer has high enough performance graphics capabilities which can be relied upon. Where this can not be guaranteed, there is the danger that the video content may be looked upon as of little use or amateurish and thus may not be taken seriously. 

2.3.2.3. AUDIO Encoding 

• Where medical acoustic phenomena (auscultation, percussion etc.) are to be portrayed, the use of audio clips, not only text or graphical representations, is welcome and important. 
• Sound tracks, as with video sequences are used only when they motivate interest or promote concentration without causing undue distraction. 
• Volume and tone can be adjusted or switched on/off. 
• Spoken sequences can optionally be displayed as written, on-screen text (such as in subtitling) and can be retrieved. Text that has already been listened to is marked as such on the display screen. 

2.3.2.4. Interaction between Different Modalities of Presentation 

• Contents, target groups and didactic concepts determine which media are best suited for the project. 
• When passive sequences require the user's extended attention, the approximate time needed for the sequence is indicated beforehand. 
• At the beginning of an application, users are informed of the media they will encounter. 
• Extensive, pre-formulated dialogues (doctor - patient etc.) are not only displayed as written text, but they are also available as audio documents and supported with pictures or videos, when appropriate. 
• The division between active elements: browsing, simulation, didactic dialogue; and passive elements: sound, video, slideshow-like presentations, reading of plain text, is discernible within the application. 
• Longer passive sequences are launched actively by the user and can be interrupted and aborted at anytime. There are several re-entry points at which the sequence can be restarted. 
• Switching between active and passive sequences occurs within a script or concept which is plausible for the user. 
• Complementary presentation of information (e. g., spoken text accompanying an animated sequence) is used to increase comprehension. In didactic applications the use of different forms of presentation is preferred to merely textual presentation where this supports comprehension. 

2.4. Ergonomics and Design 

2.4.1. Basic Requests 

• For the operation of the system/application assume no special data processing know-how is required from the user. 
• The application can be terminated from any place at any time. 
• The user interface takes into consideration the usual GUI standards, with which the users are familiar. 
• The greater the need for interaction, the more important are GUI conventions. 
• Basic functionality is self-evident - even without previous training or the consultation of the online help. 
• Mouse functions can also be initiated with keyboard combinations. 
• Rarely needed functions are able to be called by using the standard menu bar or floating palettes, depending on computer platform. 

2.4.2.Control Elements

• Icons and buttons use plausible metaphors. 
• The functionality of a control is easily understandable. 
• Features and design elements of standard software used by the target group in the everyday life (office suites, web browsers, email applications, operating system functions etc.), are, as far as possible, integrated into the design of the user interface. This includes functions like: 
• navigation tools for hypertexts 
• controls for playing audio and video 
• saving the user preferences 
• file operations 
• retrieval functions 
• mouse button functionality 
• function key allocation 
• The number and variety of the controls are limited to minimum necessary. 
• Control elements are always in the same place and have the same appearance throughout the entire application. Inactivated controls remain visible, but definitely can be detected as inactivated (e.g. dimmed or grayed out). 
• Control elements of audio-visual presentations can be intuitively operated (tape deck metaphor). 
• Clickable items can be easily recognized. 

2.4.3. Screen Layout 

• Screen layout is clear. 
• In the standardized areas of the user interface the same type of information is always found. 
• Too many concurrently opened windows are to be avoided. 

2.4.4. Color Design

• The color design of text backgrounds, entry forms and control elements is discrete and unobtrusive. They do not impair the legibility of text and the usability of picture information. 
• Color is used economically and never as an exclusive information carrier. 
• In consideration of color-blind users color combinations like red/green or blue/violet, in particular for texts and symbols, are to be avoided. Instead of mixed colors, which only differ in nuance, clearly differentiated colors are used. Framing a colored field with a black line for the reinforcement of the contrast is recommended. 
• Color symbolism is consistent. 

2.4.5. Help System

• Operating logic and control elements are described in an on-line help system. 
• Complete documentation is also available online or retrievable from the application. 
• Help texts can be printed out. 
• Help texts are hyperlinked, context sensitive and provided with an index. 
• Since on-line help systems are electronic publications, the criteria specified in this catalog apply to them as well. 
• The presence of a printed manual is not necessarily a quality criterion. However, where such a manual exists, it is understandable and carefully laid out. The manual has a clear structure with a table of contents and a glossary. 

2.5. Dialog and Didactics

• With EPMs, which are not pure tutorials, the learning components are uniquely distinguished from the other sections of the publication. 
• Learning tasks are clearly outlined detailing their content and an estimated learning time. 
• Teaching subjects are arranged in proper modules.  Learning objectives are specified for each module. 

2.5.1. Embedding of the Knowledge

• Since the knowledge to be acquired attains its meaning only by integration with known facts, the basic concepts, on which the learning process is based, are available through a hypertext environment. 
• As the progress of learning is influenced by the structure of knowledge representation, the structure of chapters, pages and paragraphs of hypertext documents are presented through an ordering principle as easily comprehensible to the learner as possible. Often a book metaphor for the presentation of textual knowledge is useful and recommended. 
• Learning strategies differ because they depend on the prior knowledge of the learner, individual learner preferences and goals. Taking this into account, the same content is accessible in different ways from within an application. 

2.5.2. Knowledge Assessment

• Learning dialogs are not limited to closed questions (as in multiple choice questions, the correct marking of  objects), but also contain open-ended questions (free text). 
• The analysis of free text input is robust, tolerant concerning spelling and also supports synonyms. 
• The evaluation of answers to open-ended questions works sufficiently correctly. 
• The evaluation of user responses is constructive, i.e. an explanation is offered, rather than negative. 
• Learning dialogs are oriented in concrete, realistic medical examination situations and use professional practice standards. 
• Typed in text can be corrected before it is analyzed by the system (first edit, then send). 
• Simulations represent a real situation as realistically as possible, focusing on the noticeable elements and processes and, if necessary, are supported by graphical or photographic data. 
• Visualizations and close-to-reality simulations are created in a way that they can be understood at the experience level of the learner. 
• Receptive sequences are connected to a special problem and are able to be worked on after being played. 
• Problem solving tasks embedded into the learning process are suitable to the topic and thus solidify as well as reinforce the newly acquired knowledge. 
• Concrete examples and case studies facilitate the learning of new concepts. 
• Knowledge assessment integrated into the learning environment gives the learner, on the one hand, feedback on the learning progress and, on the other, permits user a measure of control of the learning path. 
• The technique of knowledge assessment is adapted to the overall character of the system and, if necessary, goes beyond purely text-based questions.  For example, if interactive techniques such as simulations are applied for instructional use, then such techniques are used in the knowledge examination as well. 
• Examination of the acquired knowledge is oriented to real assessment modalities and test sequences. 
• Assessment methods consider the following aspects: 
• positive reinforcement 
• variety 
• meaningful formulation of distractors in multiple choice questions 
• 4 to 8 answer alternatives in multiple choice questions 
• progressive rate of hints to the correct solution 
• intelligent presentation of questions also dependent on the past answer behavior 
• question presentations also appear randomly as "quiz's" 
• skipping questions 
• possibility to get the correct solution 
• textual and graphical feedback on the learning progress 
• after finishing the individual learning units a evaluation summary of the session is available. 

2.5.3. Dialog / Navigation

• The orientation within the learning path is possible at any time. 
• Completed sections or modules are marked accordingly. 
• Training sessions can be interrupted, resumed or aborted at any time. After an interruption the point of reentry is easily accessible. 
• The degree of user-control depends on the target group. For beginners a "guided tour" requiring little user-control may be offered. 
• The degree of user-control also depends on the application's content. In training systems, where the comprehension of the contents requires a strict logical progression, the degree of user-control do not lead the user astray. 
• The possibility to adjust the amount of user-control from a beginner's level upwards should be part of the program design, depending on the learner's capabilities. 

2.5.4. Motivation

2.6. Extensibility

 3. Literature 

 4. Acknowledgements 

Henriette Beran (Vienna), Gui Bonsiepe (Cologne), Christoph Daetwyler (Bern), Vera Dammann (Giessen), Johannes Dietrich (Munich), Florian Eitel (Munich), Michael Esser (Eppelheim), Josef Ingenerf (Lübeck), Georg Koch (Freiburg), Ralf Köster (Göttingen), Göran Petersson (Lund), Tina Reinhard (Würzburg), Michael Schmidts (Vienna), Richard Wagner (Giessen)  

for important suggestions, ideas, material and criticism, and  

Matthias Holzer (Munich), Harold C. Lyon, Jr. (Munich), Stefan Schlachter (Freiburg), Arthur Tesche (Munich)  

for the translation.