The role of reviewers
Reviewers are pivotal in ensuring the quality of the papers and thus the conference. Reviewers provide an independent assessment of the quality of each submission. Whilst reviewers have considerable latitude, and a considerable responsibility, for interpretation of the concept of quality, we hope that the notes below will help us towards a reasonable uniformity of perception of quality standards, a fair, unbiased review process, and helpful, formative feedback for authors.
Papers under review and completed review forms are confidential and the contents are not to be revealed to other persons.
Double blind reviewing
The ascilite conferences use a double blind review process. That is, reviewers are not given the names and institutional affiliations of the authors, and authors are not given the names of the reviewers assigned to their article. If you feel that your objectivity as a reviewer has been compromised because you have identified an author, either inadvertently through routine checking of references, or other avenues, please advise the Program Committee and we will seek a replacement reviewer.
The ascilite conferences commission at least two double blind reviews for each submission. These may be supplemented, if appropriate, with another double blind review by a third reviewer, or non-blind reviews obtained from members of the Conference Committee.
Selecting and appointing reviewers
Reviewers are appointed on the basis of their expertise and experience in areas relevant for the conference. It is an honorary role, being rewarded only by acknowledgment in the online and CD versions of the Proceedings. Each ascilite conference relies to a large extent upon reviewers commissioned for previous conferences. This has helped us sustain a uniformly high standard of reviewing over the years, as most of our reviewers are 'experienced'. The ranks of 'experienced' reviewers may be supplemented from other sources, such as AJET reviewers and authors. It is not necessary for reviewers to be members of ascilite, or to be registrants for the conference. ascilite conferences also have an established policy of encouraging the induction of 'novice' reviewers, who will broaden the reviewer pool, and be in line to become the next generation of 'experienced' reviewers. This policy is facilitated by ensuring that a review allocated to a 'novice' reviewer is also allocated to 'experienced' reviewers, and is backed up by Program Committee reviews, if appropriate.
The review process: Accessing papers and forms
We anticipate that each reviewer will be allocated two to three papers, usually a mix of full and concise papers, made available on 20 August, with a due date 10 September. You will be advised by email on your login name (it will be your email address) and password for your access to the conference paper review system, via the URL http://www.ascilite.org.au/conferences/melbourne08/review/Review.php
Whilst three weeks may seem to be a tight deadline, it is similar to review process deadlines used for past conferences. Given that ascilite conferences offer authors the latest possible submission dates, it is essential that reviewers maintain good turnaround times. If you find that you must call for help and seek re-allocation of all or part of the reviewing assigned to you, it will be vitally important to inform the Program Committee sooner rather than later.
After downloading and reading the papers assigned to you, we recommend that you compose your 'Summary of contribution' and 'Detailed comments' (see below) in your word processor. Save in plain text format for doing 'copy and paste' entries during your next login to the conference paper review system. Saving in plain text format is essential in order to eliminate smart quotes, smart apostrophes, long dashes and similar special characters, because if present they cause MyReview's text entry process to lose all characters that follow an offending special character.
Applying the review criteria
You will have to use your own best judgement on the six criteria listed in the review procedure, weighted as shown. For each criterion you will be asked to select a rating from the seven point scale:
| Strong Accept | Accept | Weak Accept | Neutral | Weak Reject | Reject | Strong Reject |
You will be asked to 'self rate' on a three point 'reviewer's expertise' scale.
Next you will find two free form text entry boxes:
Summary of contribution. Please provide one to several sentences summarising your overall impression and recommendation.
Detailed comments. In plain text format, these will be the principal formative feedback. Here you should specify revisions that are to be completed to improve the quality of the paper. You could give amplifying comments and brief, illustrative examples to help authors understand the summative judgments that you have given under review criteria ratings. Please remember that the aim here is to encourage authors to improve their work, not only for this conference, but also for future conference and journal submissions. It is a section in which you can emphasise 'how you may progress...', in contrast to emphasising 'your work is bad because...'. Another aim in this section is to alert the Proceedings editors to minor or major revisions that they should check, upon receiving a revised version from the authors. Owing to production time constraints, it won't be possible to send revised versions to the original or new reviewers.
This is followed by the Yes or No question, "Candidate for the best paper award?" This item provides the Program Committee with a basis for compiling a short list for determining one to several Outstanding Paper Awards. Both 'Full' and 'Concise' papers may be eligible.
Comments for Program Committee (not shown to the authors) is another free form text entry box, where you may add any special, confidential comments for 'Program Committee eyes only', that may assist the Program Committee with the selection process, and in using its discretion when providing feedback to authors. Other matters that you may raise could include alerting the Committee to instances of excessive repetition of previously published work, or inadequate acknowledgment of the work of other writers.
Full papers and concise papers are to be reviewed using the same criteria, using your own best judgment about how well the authors have used their chosen length. The 'amount' of research represented in a concise paper may be about one third to one half the 'amount' in a full paper, but the quality is to be the same. However, with concise papers reviewers can allow a weighting towards the statement in categories of papers and proposals that concise papers are "an avenue for work in progress, for pilot studies, small scale exploratory projects, reports on highly specialised topics, or brief studies on recent developments… review key new directions for developing research based best practices and for conducting research into practices in technology supported teaching and learning".
In some cases, the reviewer may feel that it is appropriate to recommend to the Program Committee that the authors be offered a format differing from the format they nominated, e.g., an outstanding Concise paper may be given a 'Traditional' 25 minute presentation slot. However, please note that some changes of format cannot be offered. For example, changes from 'Concise' or 'Poster' to 'Full' in publication format cannot be offered. Also, changes from 'Full' to 'Concise' should be recommended only in exceptional cases where the paper could be substantially improved by a major revision with shortening. Papers submitted as 'Full' or 'Concise' which are recommended by reviewers for acceptance, but with the reviewers or the Committee changing the Presentation format to 'Poster', may be published in full if the authors so desire, upon accepting the 'Poster' offer from the Committee.